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(54) DIGITAL CAMERA 



(57)Abstract: 

PROBLEM TO BE SOLVED: To solve the problem of the compressibility of moving 
pictures caused by the presence of the limit of a memory capacity in a digital camera. 
SOLUTION: An imaging part 52 inputs the first image and the second image. A 
matching processor 14 performs pixel matching between the pictures to obtain the 
corresponding point of the grid points of a mesh arranged on the first picture on the 
second picture. The result is recorded as a corresponding point file F. An intermediate 
picture generation part 18 generates the intermediate picture between the first 
picture and the second picture based on the file F. This approximates the moving 
pictures. Since the corresponding point file is generated by the grid points, the data 
capacity is very small. 

* NOTICES * 

JPO and INPIT are not responsible for any 
damages caused by the use of this translation. 

1 This document has been translated by computer. So the translation may not reflect 
the original precisely. 

2.**** shows the word which can not be translated. 
3.1n the drawings, any words are not translated. 



CLAIMS 



[Claim(s)] 

[Claim 1]A digital camera comprising: 

An image pick-up part which photos a picture. 

A camera controller which makes an image pick-up part photo the 1st and 2nd 
pictures at the predetermined intervals, and a matching processor which calculates 
matching between the 1st picture and the 2nd picture, and outputs the result as a 
corresponding-points file. 

[Claim 2]A digital camera comprising: 

An image pick-up part which photos a picture. 

A camera controller which determines a picture of two sheets as the 1st and 2nd 
pictures among pictures photoed by image pick-up part, and a matching processor 
which calculates matching between the 1st picture and the 2nd picture, and outputs 



the result as a corresponding-points file. 

[Claim 3]A digital camera comprising: 

An image pick-up part which realizes stereo **. 

A camera controller which makes an image pick-up part photo the 1st and 2nd 
pictures that constitute a stereo image, and a matching processor which calculates 
matching between the 1st picture and the 2nd picture, and outputs the result as a 
corresponding-points file. 

[Claim 4]The digital camera according to any one of claims 1 to 3 which contains 
further an intermediate image generating part which generates an intermediate image 
of the 1st picture and the 2nd picture based on said corresponding-points file. 
[Claim 5]The digital camera according to claim 4 which contains further an indicator 
which displays the 1st picture, the 2nd picture, and an intermediate image. 
[Claim 6]A digital camera given in either of claims 4 and 5 which contain further a 
corresponding-points file attaching part which associates and records the 1st picture, 
the 2nd picture, and a corresponding-points file. 

[Claim 7]Said matching processor detects a point on the 2nd picture corresponding to 
a lattice point of a mesh provided on the 1st picture by image matching, The digital 
camera according to any one of claims 1 to 6 which defines a terminal point polygon 
corresponding to a starting point polygon which constitutes a mesh on the 1 st picture 
on the 2nd picture based on the result. 

[Claim 8]The digital camera according to claim 7 in which said matching processor 
performs matching calculation of a pixel unit based on correspondence with a singular 
point which detected by performing two-dimensional search about the 1st picture, 
and a singular point which detected by performing two-dimensional search about the 
2nd picture. 

[Claim 9]After said matching processor forms the 1 st picture and the 2nd picture into 
multiplex resolution by extracting said singular point, respectively, The digital camera 
according to claim 8 which acquires a correspondence relation of a pixel unit in a level 
whose resolution is final the finest while performing matching calculation of a picture 
unit between the same resolution levels and inheriting the result to matching 
calculation in a different resolution level. 

[Claim 10]The digital camera according to claim 1 which has further a mode setting 
part for setting up the mode of simple animation photography by said image pick-up 
part. 

[Claim 1 1]Including an image pick-up part which acquires the 1st picture and the 2nd 
picture, and a matching processor which calculates matching of the 1 st picture and 
the 2nd picture, this matching processor, A digital camera generating a 
corresponding-points file by defining a terminal point polygon corresponding to a 



starting point polygon which constitutes a mesh on the 1st picture on the 2nd picture. 
[Claim 12]An image pick-up part which photos a picture, and a camera controller 
which makes an image pick-up part photo the 1st and 2nd pictures at the 
predetermined intervals, Including a matching processor which calculates matching 
between the 1st picture and the 2nd picture, and outputs that result as a 
corresponding-points file, this matching processor, A digital camera detecting 
correspondence of the 1st picture and the 2nd picture located in the final finest 
hierarchy by hierarchizing each of the 1st picture and the 2nd picture by extracting a 
singular point, and detecting a correspondence relation of singular points from the 
hierarchy's coarser one one by one. 

[Claim 13]An image pick-up part which photos a picture, and a camera controller 
which determines a picture of two sheets as the 1st and 2nd pictures among pictures 
photoed by image pick-up part, Including a matching processor which calculates 
matching between the 1st picture and the 2nd picture, and outputs that result as a 
corresponding-points file, this matching processor, A digital camera detecting 
correspondence of the 1st picture and the 2nd picture located in the final finest 
hierarchy by hierarchizing each of the 1st picture and the 2nd picture by extracting a 
singular point, and detecting a correspondence relation of singular points from the 
hierarchy's coarser one one by one. 

[Claim 14]An image pick-up part which realizes stereo **, and a camera controller 
which makes an image pick-up part photo the 1st and 2nd pictures that constitute a 
stereo image, Including a matching processor which calculates matching between the 
1 st picture and the 2nd picture, and outputs that result as a corresponding-points file, 
this matching processor, A digital camera detecting correspondence of the 1st picture 
and the 2nd picture located in the final finest hierarchy by hierarchizing each of the 
1st picture and the 2nd picture by extracting a singular point, and detecting a 
correspondence relation of singular points from the hierarchy's coarser one one by 
one. 



DETAILED DESCRIPTION 



[Detailed Description of the Invention] 
[0001] 

[Field of the Invention]Especially this invention relates to the digital camera which 

processes predetermined in the photoed picture about a digital camera. 

[0002] 

[Description of the Prior Art]BS digital broadcasting is started, and many users come 
to enjoy the service on the Internet from PC or a gestalt telephone, and the fence of 



broadcast and communication is collapsing quickly. Video equipment and a camera 
digitize and an individual youth's information machines and equipment came to have 
broadcast communication, and very high compatibility. The "multimedia" said for a 
concept not to be in sight easily temporarily became one driving force of 
human-beings culture by technical innovation and maintenance of the infrastructure 
today when the 21st century came around. 

[0003]The digital camera which regarded the efficient preservation and printing by 
digitization of a picture at the beginning, and appeared also carries various image 
processings technique nowadays, and the function to also bear a professional youth 
with strengthening of a personal color is realized. There is no room for the digital 
instrument led by an individual to put and doubt accelerating progress of IT culture 
from now on. 
[0004] 

[Problem(s) to be Solved by the Invention]The art which generates the edge 
enhancement by a highpass filter and the picture suitable for the user s taste, such as 
a color tone conversion process, is used for some digital cameras. Art, such as 
compression by MPEG (Motion Picture Expert Group), is adopted that an animation 
should be photoed if possible for a long time. It is in such a situation and the purpose 
of this invention is in offer of the digital camera which realizes an animation by 
comparatively light data by adoption of new image processing. 
[0005] 

[Means for Solving the Problem]A mode with this invention is related with a digital 
camera. This camera uses image matching about time. That is, this camera calculates 
matching between an image pick-up part which photos a picture, a camera controller 
which makes an image pick-up part photo the 1st and 2nd pictures at the 
predetermined intervals, and the 1st picture and the 2nd picture, and contains a 
matching processor which outputs that result as a corresponding-points file. A user 
sets up "a predetermined interval" and it may be fixed beforehand. 
[0006]A camera controller photos a picture of two sheets in order an interval 
predetermined [ this ], when a user points to photography of a picture of one sheet, 
for example. Since a matching processor makes a corresponding-points file based on 
matching of a picture of two sheets, this file is behind used for it and it can generate 
an intermediate image of a picture of two sheets. As a result, an animation is 
reproducible in simple with small data volume. If a photographing interval of a picture 
of two sheets is opened to some extent, a morphing picture rather as used in the field 
of an image effect will be acquired rather than reproduction of an animation. It is also 
dramatically interesting as a function of a camera. For example, if a picture of two 
sheets is used as two persons' another face, a morphing picture of those faces is 
generable. 

[0007]Another mode of this invention is also related with a digital camera.' This 



camera is provided with the following. 

An image pick-up part which photos a picture. 

A camera controller which determines a picture of two sheets as the 1st and 2nd 
pictures among pictures photoed by image pick-up part. 

A matching processor which calculates matching between the 1st picture and the 2nd 
picture, and outputs the result as a corresponding-points file. 

A camera controller may determine a predetermined picture of two sheets as the 1 st 
and 2nd pictures in accordance with a predetermined rule among pictures which an 
image pick-up part photoed according to a user's directions. According to this mode, 
an above-mentioned morphing picture or a compression animation can be obtained 
with higher flexibility. This mode uses image matching about time, space, or both. 
[0008]Another mode of this invention is also related with a digital camera. This 
camera uses image matching about space. Namely, an image pick-up part in which this 
camera realizes stereo ** and a camera controller which makes the 1st and 2nd 
pictures from which a stereo image is constituted by an image pick-up part photo, 
Matching is calculated between the 1st picture and the 2nd picture, and a matching 
processor which outputs the result as a corresponding-points file is included. 
Therefore, a special effect image and a viewpoint change picture are generable based 
on this corresponding-points file. It is for depth information of each point of the 
picture to become clear based on corresponding points of a stereo image. 
[0009]This camera may also contain further an intermediate image generating part 
which generates an intermediate image of the 1st picture and the 2nd picture based 
on a corresponding-points file. An intermediate image is an interpolation picture 
concerning the both by time or space, and a case. It may have further an indicator 
which displays the 1st picture, the 2nd picture, and an intermediate image as an 
animation, a middle visual point picture, etc. A corresponding-points file attaching part 
which associates and records the 1st picture, the 2nd picture, and a 
corresponding-points file, for example, an IC card and other memory cards, and those 
control circuits may also be included further. 

[001 0]A matching processor may detect a point on the 2nd picture corresponding to a 
lattice point of a mesh provided on the 1 st picture by image matching, and may define 
a terminal point polygon corresponding to a starting point polygon which constitutes a 
mesh on the 1st picture in the 2nd picture top based on the result. A matching 
processor may perform matching calculation of a pixel unit between the 1 st picture 
and the 2nd picture. This calculation may follow all the pixels and may follow a lattice 
point or a lattice point, and a pixel relevant to it. 

[001 1]A matching processor may perform matching calculation of a pixel unit based 
on correspondence with a singular point which detected by performing 
two-dimensional search about the 1st picture, and a singular point which detected by 
performing two-dimensional search about the 2nd picture. In this case, after forming 



the 1st picture and the 2nd picture into multiplex resolution by extracting a singular 
point, respectively further, Matching calculation of a picture unit is performed 
between the same resolution levels, and a correspondence relation of a pixel unit in a 
level whose resolution is final the finest may be acquired, inheriting the result to 
matching calculation in a different resolution level. 

[0012]These people are application of art (henceforth "premise art") previously 
proposed by the patent No. 2927350, and are suitable for a matching method using the 
above-mentioned singular point to processing of a matching processor. However, 
premise art is not touching at all the feature of this invention about a polygon decided 
by a lattice point or it. Since a kind of simplification art of a polygon was introduced by 
this invention, size of said corresponding-points file can be made very small. 
[0013]That is, if correspondence of those pixel units is described as it is when the 1st 
and 2nd picture has a nxm pixel, respectively, combination of 2 (nxm) will arise and a 
corresponding-points file will become huge data volume. However, although it is a 
correspondence relation of lattice points, or the substantially same thing about this, 
data volume becomes less substantially by changing into describing a correspondence 
relation of the polygons decided by a lattice point. If only the 1st and 2nd picture and 
a corresponding-points file are held, an animation can be reproduced and a big effect 
will be demonstrated in transmission of an animation, storage, etc. It is the suitable art 
for a digital camera which has restriction in storage capacity of a picture. 
[0014]In premise art, this invention does not have an end as it is indispensable. 
Replace each above composition and a process arbitrarily, or between a method and a 
device, in part or all replace expression, it is added, or what changed expression into a 
computer program, a recording medium, etc. is effective as this invention. 
[0015] 

[Embodiment of the Invention] First, the image matching processing using the multiplex 
resolution singular point filtering technique and it which are used by an embodiment is 
explained in full detail as "premise art." Such art is the art in which these people have 
already acquired the patent No. 2927350, and is the best for combination with this 
invention. Since a mesh is provided on a picture and many pixels are represented with 
this invention by the lattice point, it is because the application effect over the 
matching art of a pixel unit like premise art from the first is high. However, image 
matching art employable by an embodiment is not restricted to this. The digital 
camera using premise art is concretely explained after drawing 18 . 
[0016]The [background of premise art] The component engineering of premise art 
explains in full detail by [1] first, and [2] explains procedure concretely. Furthermore, 
[3] reports the result of an experiment. 

[0017][1] Introduce the new multiple resolution filter called the details [1.1] 
introduction singular point filter of the component engineering, and calculate matching 
between pictures correctly. The preliminary knowledge about an object is 



unnecessary entirely. In each resolution, calculation of matching between pictures is 
calculated, while following the hierarchy of resolution. In that case, the hierarchy of 
resolution is followed from the coarse level in order to the minute level. A parameter 
required for calculation is thoroughly set up automatically by the dynamic calculation 
similar to human being's visual system. It is not necessary to specify the 
corresponding points between pictures with a help. 

[0018]This premise art is applicable to generation of the smooth video from 
completely automatic morphing, object recognition, stereophotogrammetry, volume 
rendering, and few frames, etc., for example. When using for morphing, the given 
picture can be transformed automatically. When using for volume rendering, the 
interim picture between sections can be reconstructed correctly. It is the same, even 
when the shape of a long distance and a section is large and the distance between 
sections changes. 

[0019][1.2] The multiplex resolution singular point filter concerning the hierarchy 
premise art of a singular point filter can save the luminosity and the position of each 
singular point which are moreover included in a picture, dropping the resolution of a 
picture. Width of a picture is set to N and height is set to M here. Below, since it is 
easy, it is assumed that it is N=M=2 n (n is a natural number). 0 and section [N] **R is 
described to be I. The pixel of the picture in (i, j) is described to be p (i j) (i, j**I). 
[0020]The hierarchy of multiplex resolution is introduced here. The hierarchized image 
group is generated by the multiple resolution filter. A multiple resolution filter 
generates another picture whose resolution is lower than the picture of even if 
two-dimensional search is performed to the picture of a basis, a singular point is 
detected and it extracts the detected singular point. Size of each picture in the m-th 
level is made into 2 m x2 m (0 <=m<=n) here. A singular point filter builds recursively the 
following four kinds of new hierarchy pictures towards falling from n. 
[0021] 
[Equation 1] 

(351) 

However, it is here, [Equation 2] 
(n,0) (»,2) (n,3) „ 



It carries out. These four pictures are henceforth called a subsidiary image (sub 
image). If min x<=t<=x+1 and max x<=t<=x+1 are described to be alpha and beta, respectively, a 
subsidiary image can be described as follows, respectively. 
[0022] 

P (m 0) =alpha(x) — alpha(y) p (m+1 ' 0) P (m 1) =alpha(x) ~ beta(y) p (m+1 ' 1) P (m ' 2) =beta(x) — 
alpha(y)p (m+1 ' 2 ) 

P (m 3) =beta(x)beta(y) p (m+1 3) , i.e., these, is considered to be a thing like a tensor 
product of alpha and beta. A subsidiary image supports a singular point, respectively. 
A singular point filter detects a singular point for every block which comprises 2x2 
pixels about a picture of a basis so that clearly from these formulas. In that case, it 
searches for a point with a maximum pixel value or the minimum pixel value about 2 
directions, i.e., length, and the side of each block. Although luminosity is adopted with 
premise art as a pixel value, various numerical values about a picture are employable. 
While a pixel from which a pixel which serves as a maximum pixel value about both 2 
directions serves as the minimum pixel value about both a maximum point and 2 
directions serves as a maximum pixel value about one side of a minimum point and 2 
directions, a pixel which serves as the minimum pixel value about another side is 
detected as a saddle point. 

[0023]A singular point filter drops resolution of a picture by representing a picture 
(here 4 pixels) of the block with a picture (here 1 pixel) of a singular point detected 
inside each block. If it carries out from a theoretical viewpoint of a singular point, 
alpha(x)alpha(y) saves a minimum point, beta(x)beta(y) saves a maximum point, and 
alpha(x)beta(y) and beta(x)alpha(y) save a saddle point. 

[0024] First, singular point filtering is independently performed to a starting point 
(sauce) picture and a terminal point (destination) picture which should take matching, 
and a series of image groups, i.e., starting point hierarchy picture, and terminal point 
hierarchy pictures are generated, respectively. A starting point hierarchy picture and 
four kinds of terminal point hierarchy pictures are generated at a time corresponding 
to a kind of singular point, respectively. 

[0025]Then, matching of a starting point hierarchy picture and a terminal point 
hierarchy picture can be taken in a series of resolution levels, and it is ********. 
Matching of a minimum point is first taken using p (m 0) . Next, based on the result, 
matching of a saddle point is taken using p (m 1) , and matching of other saddle points is 
taken using p (m 2) . And finally p (m 3) is used and matching of a maximum point is taken. 
[0026] Drawing 1 (c) and drawing 1 (d) show subsidiary image p (5 0) of drawing 1 (a) and 
drawing 1 (b), respectively. Similarly, as for drawing 1 (e) and drawing 1 (f), p C5, 1} , 
drawing 1 (g), and drawing 1 (h) show p (5 3) , respectively, as for p (5, 2) , drawing 1 (i), and 
drawing 1 G). According to the subsidiary image, matching of a characterizing portion 
of a picture becomes easy as these figures show. Eyes become clear by p (5 0) first. It is 
because eyes are the minimum points of luminosity in a face. According to p (5,1) , a 



mouth becomes clear. A mouth is because luminosity is low in a transverse direction. 
According to p (5 2) , a vertical bar of both sides of a head becomes clear. The brightest 
point of an ear or a cheek becomes clear by p (5,3) at the last. It is because these are 
the maximum points of luminosity. 

[0027]Since the feature of a picture can be extracted according to the singular point 
filter, a photographic subject reflected in a camera is discriminable by comparing the 
feature of some objects beforehand recorded as the feature of a picture photoed, for 
example with a camera. 

[0028][1.3] Write a pixel of a position (i, j) of a calculation starting point picture of a 
map between pictures to be P (n) (iJ ), and, similarly q (n) (k l) describes a pixel of a position 
(k, I) of a terminal point picture. It is considered as i, j, k, and l**L Energy 
(after-mentioned) of a map between pictures is defined. This energy is decided by a 
difference of luminosity of a pixel of a starting point picture, and luminosity of a pixel 
to which a terminal point picture corresponds, and smoothness of a map. Map f (m,0) :p (m " 
0) ->q (m ' 0) between p (m 0) which has the minimum energy in the beginning, and q (m0) is 
calculated. Based on f"- 0) , p (m ' ') with the threshold energy and map 0 between q (m ' 1) 
are calculated. This procedure continues until calculation of map f m ' 3) between p (m 3) 
and q (m 3) is completed. Each map f (mi) (i= 0, 1 and 2, — ) will be called a submap. For 
convenience of calculation of f* m0 , an order of i can be rearranged like a following 
formula. A reason to be rearranged is mentioned later. 
[0029] 
[Equation 3] 




It is sigma(i)** {0, 1 , 2, 3} here. 

[0030][1.3.1] When expressing matching between a bijection starting point picture and 
a terminal point picture by map, the map should fulfill bijection conditions between 
both images. It is because there are no superiority or inferiority on a concept in both 
images and a mutual pixel should be connected by surjection and a monomorphism. 
However, the map which should be built here unlike the usual case is the digital 
version of bijection. A pixel is specified by the lattice point in premise art. 
[0031]The map to a terminal point subsidiary image (subsidiary image provided about 
the terminal point picture) from a starting point subsidiary image (subsidiary image 
provided about the starting point picture) is expressed by f <m - s) :I/2 rr ^xI/2 
n - ra ->I/2 n - m xI/2 n - m (s= 0, 1, — ). Here, f^G, j) = (k, I) means that P (m s) (u) of a starting 
point picture is mapped by q Cm,s) (M) of a terminal point picture. Since it is easy, when f(i, 
j) = (k, I) is realized, pixel q (kJ) is described to be q KiJ) . 

[0032] Like a pixel (lattice point) treated with premise art, when data is discrete, a 
definition of bijection is important. Here, a definition is given as follows (all of i, i\ j, j\ k, 




(5*3) 



and I consider it as an integer). Each square regions first written by R in a flat surface 
of a starting point picture, [Equation 4] 

(5S4> 

******** (i= 0, — 2 m -1,j= 0, — 2 m -1). The direction of each neighborhood (edge) of 
R is defined as follows here. 
[0033] 
[Equation 5] 

(m^) (m,*) \ (m^) (m^) * (m,*) (m.a) > , (m^) "7!^ 

(S5) 



This square must be mapped by the map f by the quadrilateral in a terminal point 
image plane. The quadrilateral shown by f^^R), [Equation 6] 

9 /(iJ)^/(t+l,j)^/(.+I J+l)Qj{i+lj+l) ( ^ 6 ) 



It is necessary to fulfill ** and the following bijection conditions. 
[0034] 1. Edge of quadrilateral f m s) (R) does not cross mutually. 

2. The direction of edge of f Cm " s) (R) is equal to them of R (in the case of drawing 2 
clockwise rotation). 

3. Allow a contraction map (RITORA cushion: retractions) as relaxation conditions. 
[0035]Unless some relaxation conditions are established, it is because a map which 
fulfills bijection conditions thoroughly has only a unit map. Here, the length of one 
edge of f^'^CR) may become a triangle 0 (R), i.e., f (m s) . However, don't become a figure, 
i.e., one point, that area is set to 0, or one line segment. When drawing 2 (R) is a 
quadrilateral of a basis, although drawing 2 (A) and drawing 2 (D) fulfill bijection 
conditions, drawing 2 (B), drawing 2 (C), and drawing 2 (E) do not fill them. 
[0036]In a actual implementation, the following conditions may be imposed further that 
it should guarantee easily that a map is surjection. That is, as for each pixel on a 
boundary of a starting point picture, a pixel which occupies the same position in a 
terminal point picture ****. That is, it is f(i, j) = (i, j) (however, on four lines of i= 0, i=2 
m -1, j= 0, and j=2 m -1). This condition is also called the following "additional condition." 
[0037][1.3.2] Define energy of the cost map f about luminosity of an energy [1.3.2.1] 
pixel of a map. It is the purpose that energy looks for a map which becomes the 
minimum. Energy is mainly decided by a difference of luminosity of a pixel of a starting 
point picture, and luminosity of a pixel of a terminal point picture corresponding to it. 
That is, energy C (m s) (i j) in a point (i, j) of map f* m s) becomes settled with a following 
formula. 



[0038] 
[Equation 7] 



Here, V (p (m s) (iJ) ) and V (q^^) are the luminosity of pixel p (m s) (iJ) and q (m ' s \, }) , 
respectively. Total energy C (m ' s) of f is one valuation plan by which matching is 
evaluated, and the sum total of C (m> s) shown below can define it. 
[0039] • 
[Equation 8] 

[1.3.2.2] In order to obtain the cost smooth map about the position of the pixel for a 
smooth map, introduce another energy Df about a map. This energy is decided by the 
position of p (m s) (i j} and q (m s) Ki j) regardless of the luminosity of a pixel (i= 0, — , 2 m -1,j= 
0. — , 2 m -1). Energy D (m s) tofmapf (ms) inapoint(i j} is defined by the following formula. 
[0040] 
[Equation 9] 



(5*9 ) 



however — coefficient-parameters eta is the zero or more real numbers — 
moreover[Equation 10] 



[Equation 11] 

(a i i) 



It carries out. It is here, [Equation 12] 

ll(^y)ll = v^+F 



U12) 



It comes out, and it is and f (i\ j') decides on 0 to i'<0 and j' <0. E 0 is decided by 
distance of (i, j) and f (i, j). E 0 prevents ****(ing) a pixel to the pixel left too much. 
However, E 0 is replaced with behind another energy function. E, guarantees the 
smoothness of a map. E, expresses the distance between displacement and 



displacement of an adjacent point of p (i j} . Based on the above consideration, energy D f 
which evaluates matching and which is another valuation plan becomes settled with a 
following formula. 
[0041] 

[Equation 13] 

[1.3.2.3] The overall evaluation type concerning integration of the total calories of a 
total-calories map of a map, i.e., two or more valuation plans, is defined by lambdaC (m 
s) f +D (m ' s) f . Coefficient-parameters lambda is the zero or more real numbers here. The 
purpose is for an overall evaluation type to detect the state of taking an extremum, i.e., 
find out the map which gives the threshold energy shown with a following formula. 
[0042] 

[Equation 14] 

In the case of lambda= 0 and eta= 0, it should be cautious of a map turning into a unit 
map (that is, it becomes f^G.j) = (i, j) to all the i= 0, — f 2 m -1 and j= 0, — , 2 m -1). 
Since this premise art estimates the case of lambda= 0 and eta= 0 first like the 
after-mentioned, a map can be gradually transformed from a unit map. Supposing it 
changes the position of lambda of an overall evaluation type temporarily and defines it 
as C (m s) f +lambdaD (m s) f , The pixels where an overall evaluation type becomes only C (m 
s) f and which in the case of lambda= 0 and eta= 0 originally do not have relation in any 
way are matched only by saying [ that luminosity is only near ], and a map will become 
meaningless. Even if it transforms the map based on such a meaningless map, a 
meaning is not made at all. For this reason, it is a unit map at the start time of 
evaluation, and is considered in how to give coefficient parameters so that it may be 
chosen as the best map. 

[0043]An optical flow as well as this premise art takes into consideration a difference 
and smoothness of luminosity of a pixel. However, an optical flow cannot be used for 
conversion of a picture. It is for taking only a local motion of an object into 
consideration. A global correspondence relation is detectable by using a singular point 
filter concerning premise art. 

[0044][1.3.3] Give the determination threshold energy of a map by introduction of 
multiplex resolution, and calculate map f mjn which satisfies bijection conditions using a 
hierarchy of multiplex resolution. In each resolution level, a map between a starting 
point subsidiary image and a terminal point subsidiary image is calculated. It starts 
from the top (coarsest level) of a hierarchy of resolution, and a map of each resolution 



level is determined, taking a map of other levels into consideration. The number of 
candidates of a map in each level is restricted by using a map of a coarser level more 
highly that is,. More specifically, a map called for in a level coarser than it by one is 
imposed as a kind of constraint when determining a map in a certain level. 
[0045]first[Equation 1 5] 

(*./)-([*].[«) ( * 15) 

******** — it comes suddenly and p (mH,s) (i . tX) and q (m_1 ' s) (r ^ will be called parent of p (m 
s) (i j } and q (m,s) ( U ), respectively, [x] It is the maximum integer which does not exceed x. 
p (m s) G j) and q (m s) (i j) are called child of p (m " 1 ' s) (r r) and q (m " 1> s) (r y) , respectively. The 
function parent (i, j) is defined by following formula. 
[0046] 

[Equation 16] 

par e n<(i,i) = ([^],[|]) (^16) 

Map f m,s) between p (m> s) (i }) and q (m,s) (k 0 is determined by finding what performed energy 
calculation and became the minimum. By using f^" 1 s) (m= 1, 2, — , n), the value of f* m s) 
(i,j) = (k, I) is determined as follows. First, q (m s) (k l} imposes the conditions that it must 
be in the inside of the following quadrilateral, and narrows down what has high 
actuality among the maps which fulfill bijection conditions. 
[0047] 

[Equation 17] 



Jm^) _(t^) n (™r«) 



(ai t) 



However, it is here, [Equation 1 8] 

g^(ij) = ft m ~^(j>arent(ij)) + ^^(parenftij) + (1,1))) 

(S18) 

It comes out. In this way, the defined quadrilateral is made to call it below the 
inheritance (inherited) quadrilateral of P (m,s) (j.j). It asks for the pixel which makes 
energy the minimum in the inside of an inheritance quadrilateral. 

[0048] Drawing 3 shows the above procedure, said — a figure — setting — the 
starting point — a picture — A — B — C — D — a pixel — the — m - one — a level 
— setting — respectively — a terminal point — a picture — A — ' — B — ' — C — 
****(j ng ) _ having . Pixel p (m s) (i ^must be ****(ed) to pixel q (m - s) Km) G ^ 
which exists in the inside of inheritance quadrilateral A'B'C'D'. The mediation to the 
map of the m-th level from a map of the m-1st levels is made by the above 



consideration. 

[0049]Energy E 0 defined previously is transposed to a following formula, in order to 

calculate submap f (m 0 ) in the m-th level. 

[0050] 

[Equation 19] 

£b w) = n/ (m ' o) ^i)-5 (ra) (i,i)ir (5S1 9) 



A following formula is used in order to calculate submap f m s) 
[0051] 

[Equation 20] 

= \\f im *(iJ) - f {nL ' S - l) (iJW (1 < 0 



(S20) 



In this way, the map which maintains the energy of all the submaps at a low value is 
obtained. The submap corresponding to a different singular point is associated within 
an identical level so that the similarity of submaps may become high by the formula 20. 
The formula 19 shows distance with the position of the point that f^'^ G, j), and (i, j) at 
the time of considering it a part of pixel of the m-1st levels should be projected. 
[0052]The following measures are taken when a pixel which fulfills bijection conditions 
does not exist in an inside of inheritance quadrilateral A'B'C'D' temporarily. First, 
distance from a boundary line of A'B'C'D 1 investigates a pixel which is L (it is L= 1 in 
the beginning). Among those, if that from which energy becomes the minimum fulfills 
bijection conditions, this will be chosen as a value of f^'^G, j). L is enlarged until such 
a point is discovered or L reaches L (m) max of the maximum. L (m) max is immobilization 
to each level m. When such a point is not discovered at all, map that the 3rd condition 
of bijection is disregarded temporarily and area of a quadrilateral of a converting 
destination becomes zero is also accepted, and f^^G, j) is determined. When a point 
of still fulfilling conditions is not found, the 1st and 2nd conditions of bijection are 
removed next. 

[0053]Avoiding that a map is influenced by details of a picture, an approximation using 
multiplex resolution is indispensable in order to determine a global correspondence 
relation between pictures. If an approximation by multiplex resolution is not used, it is 
impossible to find out a correspondence relation between pixels with a far distance. In 
that case, size of a picture must be limited to a very small thing, and only a small 
picture of change can be treated. A correspondence relation between such pixels is 
made hard to find, in order to usually require smoothness of a map. It is because 
energy of a map to a pixel with distance from a pixel is high. According to the 
approximation using multiplex resolution, a suitable correspondence relation between 
such pixels can be found out. Those distance is because it is small in an upper level 



(coarse level) of a hierarchy of resolution. 

[0054][1.4] There is difficulty of one PAREMETA adjustment of the main faults of 
matching art of automatic determination existing of optimal PAREMETA value. When 
the most, it is very difficult for adjustment of a parameter to be performed by help 
work and to choose optimal value. According to a method concerning premise art, 
automatic determination of the optimal parameter value can be carried out thoroughly. 
[0055]A system concerning premise art contains 2 PAREMETA, and lambda and eta. 
Speaking directly, lambda's being the dignity of a difference of luminosity of a pixel and 
eta's showing the rigidity of a map. An initial value is 0, it fixes to eta= 0 first and a 
value of these parameters makes lambda increase from 0 gradually. When making a 
value of an overall evaluation type (formula 14) into the minimum moreover, enlarging 
a value of lambda, generally a value of C (m s) f about each submap becomes small. This 
means that 2 pictures must match more fundamentally. However, if lambda exceeds 
an optimum value, the following phenomena will occur. 

[0056] 1. The pixels which should not correspond essentially are accidentally matched 
only by saying [ that luminosity is only near ]. 

2. As a result, a correspondence relation of pixels becomes amusing and a map begins 
to collapse. 

[0057]3. As a result, in the formula 14, D (m s) f tends to increase rapidly. 
4. As a result, since a value of the formula 14 tends to increase rapidly, f* m,s) changes 
so that a rapid increase in D (m s) f may be controlled, and C (m s) f increases. 
Therefore, maintaining the state of taking the minimum making lambda increase, C Cm ' s) 
f detects a threshold changed to an increase from reduction, and the formula 14 
makes the lambda an optimum value in eta= 0. Next, eta is increased little by little, an 
action of C {m s) f is inspected, and automatic determination of eta is carried out by a 
below-mentioned method, lambda is also decided corresponding to the eta. 
[0058]This method resembles operation of a focal mechanism human beings visual 
system. In human being's visual system, matching of a picture of right-and-left both 
eyes is taken, moving one eye. The eye is fixed when an object can recognize clearly. 
[0059][1.4.1] It is increased from 0 by the dynamic determination lambda of lambda by 
predetermined unit width, and a submap is evaluated whenever a value of lambda 
changes. Total calories are defined by lambdaC (m s) f +D (m s) f like the formula 14. D Cm s) f of 
the formula 9 expresses smoothness, and, in a unit map, becomes the minimum 
theoretically, and E 0 and E, also increases it, so that a map is distorted. Since E, is an 
integer, the minimum unit width of D (m s) f is 1 . For this reason, if change (decrement) of 
the present lambdaC (ms) (i fj) is not one or more, total calories cannot be reduced by 
changing a map. Because, it is for total calories not decreasing, unless lambdaC (ms) (i j) 
decreases or more by one, since D (m s) f increases one or more with change of a map. 
[0060]With a basis of this condition, and an increase in lambda, when normal, it is 
shown that C (ms) {ij) decreases. A histogram of C (m s) (i j} is described to be h (I), h (I) is 



the number of pixels whose energy C (m,s) (iJ) are I 2 . Since lambdal 2 >=1 is realized, a case 
of l 2 =1 /lambda is considered, for example. When lambda carries out fine amount 
change from lambda! to lambda 2 , [Equation 21] 

l ^" J rl- jrAi i r A 2 h(l) 



(S21) 



It comes out and A pixels shown, [Equation 22] 

It changes [ rather than ] to a stable state with ** energy. Here, temporarily, if all of 
the energy of these pixels become zero, it approximates. The value of C (m s) f this 
formula, [Equation 23] 

dc {m,s) = _A U23) 

Carrying out **** change is shown and it is the result,[Equation 24] 
dCi m " ] h{l) 

It ******. h (I) Since it is >0, C (m s) f usually decreases. However, when lambda tends to 
exceed an optimum value, an above-mentioned phenomenon, i.e., the increase in C (m s) 
f , occurs. The optimum value of lambda is determined by detecting this phenomenon. 
[0061]When making H (h> 0) and k into a constant,[Equation 25] 

h{i) = Ht k = ^ (32 5> 

It is if it assumes, [Equation 26] 



ax ~ A 5 ' 2 **/ 2 <^ 2 6 >. 

**********. If it is k!=-3 at this time, [Equation 27] 

H 

°f =C+ (3/2 + k/2)X*W (^2 7) 

It becomes. This is a general formula of C Cm s) f (C is a constant). 

[0062]When detecting the optimum value of lambda, safety may be seen further and 
the number of the pixels which break bijection conditions may be inspected. When 
determining the map of each pixel here, the probability of breaking bijection conditions 
is assumed to be p 0 . In this case [Equation 28] 



dA _ k{l) 

ax " a*/* 



(2t 2 8) 



Since it is ******(ing), the number of pixels which break bijection conditions 

increases at a rate of a following formula. 

[0063] 

[Equation 29] 



(5*2 9) 



It follows, [Equation 30] 

M(0 =1 <* 30 > 

It is ******. the time of assuming h(l) =Hl k temporarily — for example [Equation 31] 
B^W* = (43 1) 

It becomes ******. However, if lambda exceeds an optimum value, the upper value will 
increase quickly. This phenomenon can be detected, it can be inspected whether the 
value of B 0 lambda 3/2+k/2 /2 m exceeds abnormal value B^,,^ and the optimum value of 
lambda can be determined. Rate-ofHncrease B, of a pixel which breaks the 3rd 
condition of bijection is checked by similarly inspecting whether the value of 
lambda 372 4k/2 /2 m exceeds abnormal value B lthres . The reason for introducing factor 2 
m is mentioned later. This system is not sensitive to these two thresholds. By 
observation of energy C (m s) f , these thresholds can be used in order to detect the 
excessive distortion of the map which it failed to detect. 

[0064]In an experiment, when calculating submap f* m ' s) , if lambda exceeded 0.1, 
calculation of f* m ' s) was stopped and shifted to calculation of f <m * s+1) . At the time of 
lambda> 0.1, it is because [ in luminosity 255 level of a pixel ] a difference in "3" 
influenced calculation of a submap only, and it is because it was difficult to obtain a 
right result at the time of lambda> 0.1. 
[0065][1 .4.2] Histogram h (I) 

It does not depend for an inspection of C (m s) f on the histogram h (I). In the case of an 
inspection of bijection and its 3rd condition, influence can be received in h (I). When it 
actually (lambda, C (m s) f ) plots, k is usually in the one neighborhood. In an experiment 
B 0 lambda 2 and BJambda 2 were inspected using k= 1. If a true value of k is less than 
one, B 0 lambda 2 and Bjlambda 2 will not become a constant but will increase gradually 
according to factor lambda 0-10 ' 2 . If h (I) is a constant, factors are lambda 1/2 , for 
example. However, such a difference is absorbable by setting up threshold B^^ 
correctly. 



[0066]A center assumes a starting point picture here to be (x 0 , y 0 ) f and a circular 
object of the radius r like a following formula. 
[Equation 32] 



On the other hand, a terminal point picture presupposes that a center (x 1? y^ and a 
radius are the objects of r like a following formula. 
[Equation 33] 



c(x) presupposes here that it is a form of c(x)=x k . In the histogram h (I), a center (x 0 , y 0 ), 

and (x 1t y,) serve as a form of a following formula, when far enough. 

[Equation 34] 
h(l) oc rl k (Jfc ^ 0) 

A picture shows an object with a clear boundary line embedded for a background at 
the time of k= 1. This object becomes bright as a center is dark and goes to the 
circumference. A picture expresses an object with an ambiguous boundary line at the 
time of k=-1. This object becomes dark as a center is the brightest and goes to the 
circumference. Even if it thinks that there is a general object in the middle of these 2 
types of object, generality is not very lost. Therefore, k can cover a case where it is 
the most as -1<=k<=1, and it is secured that the formula 27 is generally a decreasing 
function. 

[0067]As shown in the formula 34, it should be cautious of r being influenced by 
resolution of a picture, i.e., r is proportional to 2 m. For this reason, 2 m of factors 
were introduced in [1.4.1]. 

[0068][1.4.3] The automatic determination of the dynamic determination parameter 
eta of eta can be carried out in a similar way. It is first referred to as eta= 0, and final 
map f^and energy C (n) f in the finest resolution are calculated. Continue, only a certain 
value deltaeta makes eta increase, and last map f^and energy C (n) f in the again finest 
resolution is recalculated. This process is continued until an optimum value can be 
found, eta shows the rigidity of a map. It is because it is the dignity of a following 
formula. 




(5£3 2) 




(5*3 3) 



[0069] 

[Equation 35] 



(5*3 5) 



When eta is 0, D (n) f will be determined regardless of the last submap, will be elastically 
transformed by the present submap, and will be distorted too much. On the other hand, 
as for D (n) f , eta is decided by the last submap nearly thoroughly at the time of a very 
big value. At this time, a submap has dramatically high rigidity, and a pixel is projected 
on the same place. As a result, a map turns into a unit map. When the value of eta 
increases gradually from 0, C (n) f decreases gradually like the after-mentioned. 
However, energy begins to increase as it is shown in drawing 4 , when the value of eta 
exceeds an optimum value. The X-axis of the figure is eta and a Y-axis is C f . 
[0070]The value of optimal eta that makes C (n) f the minimum by this method can be 
obtained. However, as a result of various elements' influencing calculation compared 
with the case of lambda, C (n) f changes, swinging small. In eta, it is because all the 
submaps are recalculated but as a submap is recalculated once, whenever an input 
carries out fine amount change of the case of lambda. For this reason, it cannot be 
judged immediately whether the value of obtained C (n) f is the minimum. If the 
candidate of the minimum is found, it is necessary to look for the true minimum by 
setting up the still finer section. 

[0071][1.5] When determining a correspondence relation between super sampling 
pixels, in order to increase flexibility, a range of f^^is extensible to RxR (R is a set of 
the real number). In this case, luminosity of a pixel of a terminal point picture is 
interpolated and it is a nonintegral point, [Equation 36] 



f m ' s) with the luminosity which can be boiled and set is provided. That is, super 
sampling is performed. In an experiment, f m s) is allowed to take an integer and a 
halfHntegral value, [Equation 37] 



A " J (SC 3 7) 



) + (0.5,0.5), 



**,[Equation 38] 

(V(<&?>) + n9( ( ,"# (l ,i)))/2 «,., 

Be alike was given. 

[0072][1.6] When a normalization starting point picture and a terminal point picture of 
luminosity of each picture contain an extremely different object, it is hard to use 
luminosity of the original pixel for calculation of a map as it is. [ of a pixel ] since the 
difference of luminosity is large, energy C (m s) f about luminosity becomes large too 
much — right evaluation — butter fish — ** — a sake — it is . 



[0073]For example, a case where matching of people's face and a face of a cat is 
taken is considered. A face of a cat is covered with hair and a very bright pixel and a 
very dark pixel are mixed. In this case, in order to calculate a submap between 2 faces, 
a subsidiary image is normalized first. That is, luminosity of the darkest pixel is set as 
0, the brightest it is set as 255, and it asks for luminosity of other pixels by linear 
interpolation. 

[0074][1.7] Use how calculation advances linearly according to a scan of an 
implementation starting point picture inductive. A value of f m s) is determined about 
pixel [ of a left end on introduction and No. 1 ] (i, j) = (0, 0). Next, a value of each f Cm> s) 
(i, j) is determined, increasing i every [ 1 ]. When a value of i reaches width of a picture, 
a value of j is increased one and i is returned to 0. Henceforth, f m,,) (i t j) is determined 
with a scan of a starting point picture. If correspondence of a pixel is decided about all 
the points, one map f* m s) will be decided. 

[0075]If corresponding-points q^is decided about certain p (i j)f corresponding-points 
^fCof pG.j+i)](u+i) w '" b e decided next. Under the present circumstances, a position of q Ki 
j+1) is restricted by position of q^ ^in order to fulfill bijection conditions. Therefore, in a 
point that corresponding points are decided previously, in this system, a priority 
becomes high. An excessive deviation will be added to the last map called for if the 
state where priority is always (0, 0) given most continues. With this premise art, in 
order to avoid this state, f^^is decided by the following methods. 
[0076]When it is 0 first (second mod 4), it decides making (0, 0) into the starting point 
and increasing i andj gradually. When (second mod 4) is 1, a right end point of the top 
line is made into the starting point, and i is decided, making reduction andj increase. 
When (second mod 4) is 2, a right end point of a lowermost row is made into the 
starting point, and it decides, decreasing i andj. When (smod 4) is 3, a left end point of 
a lowermost row is made into the starting point, and i is decided, decreasing an 
increase andj. Since the concept s of a submap, i.e., a parameter, did not exist in the 
n-th level with the finest resolution, even if it was s= 0 and s= 2, 2 directions were 
calculated continuously. 

[0077]In a actual implementation, a value of f <m - s) (i, j) (m= 0, — , n) which fulfills 
bijection conditions as much as possible out of a candidate (k, I) was chosen by giving 
a penalty to a candidate who breaks bijection conditions, phi is hung on the energy D 
of a candidate who breaks the 3rd condition (k, I), and, on the other hand, psi is hung 
on a candidate who breaks the 1st or 2nd condition. phi= 2 and psi= 100000 were used 
this time. 

[0078]The following tests were done when determining =f Cm s) (i, j) as a procedure 
actual for a check of the above-mentioned bijection conditions (k, I). That is, it is 
confirmed whether z ingredient of an outer product of a following formula becomes 
zero or more to each lattice point (k, I) contained in an inheritance quadrilateral of 

S) (U). 



[0079] 

[Equation 39] 

WmXxB (*..> 

However, it is here, [Equation 40] 



[Equation 41] 



(SC4 1 ) 



Come out and it is (a vector is made into a three-dimensional vector here, and the 
z-axis is defined in a rectangular right-handed coordinate system). If W is negative, it 
will be made whether a penalty is given and is made and not to choose by hanging psi 
on D (m s) (k l) about the candidate. 

[0080] Drawing 5 (a) and drawing 5 (b) show the reason for inspecting this condition. 
Drawing 5 (a) expresses a candidate without a penalty, and the candidate to whom 
drawing 5 (b) has a penalty, respectively. When determining map f mi s) (i, j+1) to an 
adjacent pixel (i,j+1), if z ingredient of W is negative, the pixel which satisfies bijection 
conditions on a starting point image plane does not exist. Because, it is for q^Vo 
exceeding the boundary line of the adjoining quadrilateral. 

[0081][1.7.1] In an order implementation of a submap, when the number of resolution 
levels was even, sigma(O) =3, sigmaO) =2, sigma(2) =1, sigmaO) =0, and sigma(4) =3 
were used using sigma(O) =0, sigmaO) =1 , sigma(2) =2, sigma(3) =3, and sigma(4) =0 at 
the time of odd number. By this, a submap was shuffled moderately. Originally a 
submap is four kinds and s is either 0-3. However, processing which is actually 
equivalent to s= 4 was performed. The reason is mentioned later. 
[0082][1.8] After a map between an interpolation calculation starting point picture and 
a terminal point picture is determined, luminosity which is a pixel which corresponds 
and suits is interpolated. Try linear interpolation was used in an experiment. Square p 

in a starting point image plane, (i. j) It is assumed that P(i + i.j)P(i.j + i)P(i + i.j + o is projected on quadrilateral q ffiij) q Ki+ i. 

j) Qfaj+D <fo+i.j+i) on a terminal point image plane. Since it is easy, distance between 
pictures is set to 1. The pixel r (x, y, t) of an intermediate image whose distance from 
a starting point image plane is t (0<=t<=1) (0 <=x<=N-1, 0 <=y<=M-1) is called for in 
the following ways. A position (however, x f y, t**R) of the pixel r (x, y, t) is first 
searched for with a following formula. 
[0083] 

[Equation 42] 



+ dx(l -JifKl-t)(i + lJ)+<t*0- Li) 
+ (1 - </z)<fy(l - Ofti + I) + <1 - dx)dytf(i 1 j + I) 
+ - t)(i + l,i + 1) + dxdytf{i + l,i + 1) 

(*4 2) 

The luminosity of the pixel in r (x, y, t) is continuously determined using the following 

formula. 

[0084] 

[Equation 43] 

+ cfa(i-ifc)<i-ov(^ij)) + ^-*W«/(^ij)) 

+ (1 - - <)V(p (ii#+ i)) + (1 - <k)*«Vfo (w) ) 

+ <£x<iy{l -t)V(p Ci+u+1) ) + ^iiytV(^/{,>i^».i)) 

(5*4 3) 

dx and dy are parameters and change from 0 to 1 here. 

[0085][1.9] The determination of the map in case a constraint does not exist at all 
until now [ map ] when a constraint is imposed was described. However, when 
correspondence relations are beforehand specified between the specific pixels of a 
starting point picture and a terminal point picture, a map can be determined after 
making this into a constraint. 

[0086]By the rough map which moves the specific pixel of a starting point picture to 
the specific pixel of a terminal point picture first, a fundamental idea transforms a 
starting point picture roughly, and calculates the map f correctly after an appropriate 
time. 

[0087]It opts for a rough map which projects a specific pixel of a starting point picture 
on a specific pixel of a terminal point picture first, and projects other pixels of a 
starting point picture on a suitable position. That is, a pixel near a specific pixel is the 
map which is projected near the place where the specific pixel is projected. A rough 
map of the m-th level is described to be F* m) here. 

[0088]It opts for the rough map F in the following ways. First, a map is specified about 
some pixels. It is a pixel of n s individual about a starting point picture, [Equation 44] 

(*44) 

The following values are decided when specified. 
[Equation 45] 



(34 50 



The amount of displacement of other pixels of a starting point picture is an average 
which makes weighting displacement of p (jh jh) (h= 0, — , n s -1), and is called for. That is, 
pixel p (i j} is projected on the following pixels of a terminal point picture. 
[0089] 

[Equation 46] 



2 n—m 



(5*4 6) 



However, it is here,[Equation 47] 

* total tveighi{i,j) (S4 7) 



[Equation 48] 

h=n,— l 

total weighty j) = £ l/\\(i k - i,j k _ ,)||» 4 8 ) 



It carries out. 

[0090]It continues, and energy D (i D of the map f is changed so that the candidate 
map f near F^may have less energy. Correctly, D (m s) (i j} is, [Equation 49] 



(34 9) 



It comes out. It corrects,[Equation 50] 



if ||f<->(i,j) - /"^(i-iMI 2 < l^yJ 

IM*"^*, j) - /^H*, otherwise 

(S50) 



It comes out, and it is and is referred to as kappa and rho>=0. f is thoroughly 
determined as the last by the automatic calculation process of the above-mentioned 
map. 

[0091]Here, in f" 1 s) (i, j), when close to F <m) (i, j) enough that is, those distance, 



[Equation 51] 

[" 2 2(£-m)] (Si 5 1) 

When it is less than, it should be cautious of E 2 Cm ' s) being set to 0. The reason 
defined such is because he would like to decide the value automatically so that it may 
settle in a suitable position in a terminal point picture, as long as each f^'^Gj) is close 
to F^G, j) enough. It is automatically mapped so that it is not necessary to specify an 
exact correspondence relation in detail and a starting point picture may match a 
terminal point picture for this reason. 

[0092][2] Explain a flow of processing by each component engineering of concrete 
procedure [1]. Drawing 6 is a flow chart which shows the whole premise technical 
procedure. As shown in the figure, processing which used a multiplex resolution 
singular point filter first is performed (S1), and matching of a starting point picture and 
a terminal point picture is taken continuously (S2). However, S2 may process image 
recognition etc. based on the feature of a picture acquired by S1 rather than is 
indispensable. 

[0093] Drawing 7 is a flow chart which shows details of S1 of drawing 6 . Here, it is 
premised on taking matching of a starting point picture and a terminal point picture by 
S2. Therefore, first, with a singular point filter, a starting point picture is hierarchized 
(S10) and a series of starting point hierarchy pictures are acquired. A terminal point 
picture is continuously hierarchized in a similar way (S11), and a series of terminal 
point hierarchy pictures are acquired. However, an order of S10 and S11 is arbitrary, 
and can also generate a starting point hierarchy picture and a terminal point hierarchy 
picture in parallel. 

[0094] Drawing 8 is a flow chart which shows details of S10 of drawing 7 . Size of a 
starting point picture of a basis is made into 2 n x2 n . Since a starting point hierarchy 
picture is made sequentially from the one where resolution is finer, it sets to n the 
parameter m which shows a resolution level which is the target of processing (S100). 
Continuously Picture p (m 0) of the m-th level, p (m p (m - 2) t A singular point is detected 
from p (m ' 3) using a singular point filter (S101), and picture p (m " 1 0) of the m-1st levels, 
p (m-i,n p (m-i .2) and p (m-i.3) are generatec j respectively (S102). Here, since it is m=n, it is 
p (m.o) =p(m .i) =p (m.2) =p (m.3) =p (n) and four kjnds of subsidiary j mages are generated from 
one starting point picture. 

[0095] Drawing 9 shows correspondence relation between a part of picture of the m-th 
level, and a part of picture of the m-1st levels. A numerical value of the figure shows 
luminosity of each pixel. p (ms) of the figure symbolizes four pictures of p (m0) - p (m - 3) , and 
in generating p ( ^ 10 \ it thinks that p (m s) is p {m 0) . Under a rule shown by [1.2], p (m - 10) 
about a block as which luminosity was filled in for example, with the figure. p (m_1 " 1} 
replaces "8" and p (m_1 • 2) among 4 pixels "3" contained there by one pixel which 



acquired "10" for "6" and p (m ~ 1,3) , respectively, and acquired this block, respectively. 
Therefore, size of a subsidiary image of the m-1st levels becomes 2 m_1 x2 m ~ 1 . 
[0096]The decrement of the m is carried out continuously (S103 of drawing 8 ), and it 
checks that m is not negative (S104), it returns to S101, and then a subsidiary image 
with coarse resolution is generated. As a result of this repetition processing, when a 
subsidiary image of m= 0, i.e., the 0th level, is generated, S10 is completed. Size of a 
subsidiary image of the 0th level is 1x1. 

[0097] Drawing 10 has illustrated a starting point hierarchy picture generated by S10 
about a case of n= 3. Only the first starting point picture is common to four series, and 
a subsidiary image is henceforth generated independently according to a kind of 
singular point, respectively. Processing of drawing 8 is common also to S1 1 of drawing 
7, and a terminal point hierarchy picture is also generated through same procedure. 
Processing by S1 of drawing 6 is completed above. 

[0098]In premise art, in order to progress to S2 of drawing 6 , matching evaluation is 
prepared. Drawing 1 1 shows the procedure. As shown in the figure, two or more 
valuation plans are set up first (S30). Energy C (m ' s) f about a pixel introduced by 
[1.3.2.1] and energy D (m s) f about smoothness of a map introduced by [1.3.2.2] are it. 
Next, these valuation plans are unified and an overall evaluation type is stood (S31). If 
eta which total-calories lambdaC (m ' s) f +D (m s) f introduced by [1.3.2.3] is it, and was 
introduced by [1.3.2.2] is used, sigmasigma (lambdaC (m - s) (i j} +etaE 0 (m s) (i j} +E 1 (m ' s) (i j5 ) 
(formula 52) 

It becomes. However, total calculates by 0, 1 — , and 2 m -1 about i and j, respectively. 
Preparation of matching evaluation by the above is completed. 

[0099] Drawing 12 is a flow chart which shows details of S2 of drawing 6 . As [1] 
described, matching of a starting point hierarchy picture and a terminal point 
hierarchy picture is mutually taken by the pictures of the same resolution level. In 
order to take global matching between pictures good, matching is calculated 
sequentially from a level with coarse resolution. Since a starting point hierarchy 
picture and a terminal point hierarchy picture are generated using a singular point 
filter, a position and luminosity of a singular point are clearly saved also on a level with 
coarse resolution, and a result of global matching becomes what was dramatically 
excellent compared with the former. 

[0100]Like drawing 1 2 , 0 and the level parameter m are first set as 0 for 
coefficient-parameters eta (S20). Continue and matching is calculated between each 
of four subsidiary images of the m~th level in a starting point hierarchy picture, and 
four subsidiary images of the m~th level in a terminal point hierarchy picture, It asks 
for 4 kinds of submap f <m,s) (s= 0, 1, 2, 3) which fulfills bijection conditions, respectively 
and makes energy the minimum (S21). Bijection conditions are inspected using an 
inheritance quadrilateral described by [1.3.3]. Under the present circumstances, since 
a submap in the m-th level is restrained by them of the m-1st levels as shown in the 



formulas 17 and 18, matching in a level with coarser resolution is used one by one. 
This is the vertical reference between different levels. Although it is m= 0 now and 
there is no level coarser than it, this exceptional processing is later mentioned by 
drawing 1 3 . 

[0101]On the other hand, horizontal reference in an identical level is also performed, 
like the formula 20 of [1 .3.3] — 1) decides f< m 3) is similar to &• 2) , and f (m ' 2) is similar to 
f* m ' 1) , and similar to f (n, 0) f respectively. This is because a situation where submaps 
completely differ is unnatural, if they are contained in the starting point picture same 
from the first and a terminal point picture at all even if a kind of singular point is 
different. Energy becomes small, so that submaps are near, as shown in the formula 20, 
and it considers that matching is good. 

[0102]Since there is no submap which can be referred to on level same about f (m,0) 
which should be decided first, as shown in the formula 19, a level coarse [ one ] is 
referred to. However, in an experiment, after being found to f* m " 3) , procedure of 
updating f <m * 0) once by making this into a constraint was taken. It is equal to this 
substituting s= 4 for the formula 20, and making f (m,4) into new f^ 05 . It is for avoiding a 
tendency for a degree of association of f (m,0) and f* m,3) to become low too much, and an 
experimental result became fitness more by this measure. In addition to this measure, 
in an experiment, a shuffle of a submap shown in [1.7.1] was also performed. This is 
also the meaning which keeps close a degree of association of the submaps originally 
decided for every kind of singular point. In order to avoid a deviation depending on the 
starting point of processing, a point of changing a position of the starting point 
according to a value of s is as [1.7] having described. 

[0103] Drawing 13 is a figure showing signs that a submap is determined in the 0th 
level. Since each subsidiary image merely comprises one pixel, it is automatically 
decided on the 0th level that all four submap * 0,8> will be unit maps. Drawing 14 is a 
figure showing signs that a submap is determined in the 1 st level. A subsidiary image is 
constituted from 4 pixels by the 1st level, respectively. These 4 pixels are shown by 
solid line in the figure. The following procedures are stepped on when looking for 
corresponding points of the point x of p (1 s) in q (1 s) now. 

[0104]1. Search for the top left point a of the point x, the upper right point b, the lower 
left point c, and the lower right point d in resolution of the 1st level. 

2. One point a-d looks for a pixel which belongs in a coarse level, i.e., the 0th level. In 
the case of drawing 14 , point a-d belongs to pixel A-D, respectively. However, pixel 
A-C is a virtual pixel which originally does not exist. 

3. Plot corresponding-points A f of pixel A~D which is the 0th level and has already 
been found - D' in q 0, s) . Pixel A T - C shall be virtual pixels, and shall be in the 
respectively same position as pixel A-C. 

4. Consider that corresponding-points a of the point a in the pixel A is in pixel A', and 
plot point a'. At this time, it is assumed that a position which point a occupies in pixel 



A' is the same as a position (in this case, lower right) which the point a occupies in the 
pixel A. 

Corresponding-points b' - cT are plotted by the same method as 5.4, and an 
inheritance quadrilateral is made from point a - d\ 

6. Look for corresponding-points x' of the point x so that energy may become the 
minimum in an inheritance quadrilateral. As a candidate of corresponding-points x\ 
the center of a pixel may limit to what is contained in an inheritance quadrilateral, for 
example. In the case of drawing 14 , all of four pixels become a candidate. 
[0105]It is a decision procedure of corresponding points of the point x with the above. 
Same processing is performed about other points of all the, and it opts for a submap. 
In a level [ 2nd / more than ] level, since it is thought that an inheritance quadrilateral 
loses its shape gradually, a situation where an interval of pixel A' - D' is vacant as 
shown in drawing 3 occurs. 

[0106]In this way, if four submaps of a certain m-th level are decided, m will be 
**************ed (S22 of drawing 12 ), and it will confirm that m is not over n (S23), 
and will return to S21. It asks for a submap of a level of resolution gradually fine 
whenever it returns to S21 hereafter, and when it finally returns to S21, map f* n) of the 
n-th level is decided. Since this map becomes settled about eta= 0, it writes it as f 00 
(eta= 0). 

[0107]Only deltaeta shifts eta in order to ask also for a map about next different eta, 
and the zero clear of the m is carried out (S24). It checks that new eta is not over 
predetermined search cancel value eta max (S25), returns to S21, and asks for map f* n> 
(eta=deltaeta) about this eta. This processing is repeated and it asks for f 00 
(eta=ideltaeta) (i= 0, 1, — ) by S21. When eta exceeds eta max , it progresses to S26, and 
optimal eta=eta opt is determined by a below-mentioned method, and f* n) (eta=eta opt ) is 
eventually made into map f* n) . 

[0108] Drawing 15 is a flow chart which shows details of S21 of drawing 12 . With this 
flow chart, a submap in the m-th level is decided about certain fixed eta. When opting 
for a submap, with premise art, optimal lambda is independently decided for every 
submap. 

[0109]As shown in the figure, the zero clear of s and lambda is carried out first (S210). 
Next it asks for submap f (m,s) which makes energy the minimum about lambda at that 
time (it reaches and is about eta to dark) (S211), and this is written to be s) 
(lambda= 0). Only deltalambda shifts lambda in order to ask also for a map about 
different lambda, and it checks that new lambda is not over predetermined search 
cancel value Iambda max (S213), returns to S211, and asks for f°"' s) 
(lambda=ideltalambda) (i= 0, 1, — ) by subsequent repetition processings. When lambda 
exceeds lambda max , it progresses to S214, and optimal lambda^ambda^ is determined, 
and f^Oambda^ambda^) is eventually made into map f m s) (S214). 
[01 10]Next t the zero clear of lambda is carried out in order to ask for other submaps 



in an identical level, and s is **************ed (S215). It checks that s is not over 4 
(S216), and returns to S21 1. If set to s= 4, like ****, f" 0) will be updated using f"' 3 \ 
and determination of a submap in the level will be ended. 

[01 1 1] Drawing 16 is a figure showing an action of energy C (m s) f corresponding to f Cm,s) 
(lambda=ideltalambda) (i= 0, 1, — ) called for while changing lambda about certain m 
and s. If lambda increases, C (m s) f will usually decrease, as [1.4] described. However, if 
lambda exceeds an optimum value, C (m,s) f will start to increase. So, with this premise 
art, lambda in case C (m s) f takes the minimal value is decided to be lambda opt . What is 
necessary is just to observe the first minimum point, in order for a map to already 
have collapsed at the time and not to make a meaning, even if C (m s) f becomes small 
again in the range of lambda>lambda opt , as shown in the figure. lambda opt is 
independently decided for every submap, and, finally becomes settled one also about 

f(n) 

[0112]On the other hand, drawing 17 is a figure showing an action of energy C (n) f 
corresponding to f* n) (eta=ideltaeta) (i= 0, 1, — ) called for while changing eta. If eta 
increases also here, C (n) f will usually decrease, but if eta exceeds an optimum value, 
C (n) f will start to increase. Then, eta in case C (n) f takes the minimal value is decided to 
be eta opt . Drawing 1 7 may consider a figure to which near the zero of a horizontal axis 
of drawing 4 was expanded. If eta opt is decided, terminal decision of the f* n) can be 
carried out. 

[01 13]As mentioned above, according to this premise art, various merits are obtained. 
Since it is not necessary to detect edge probably, a technical problem of edge 
detection type conventional technology is cancelable. Transcendental knowledge over 
an object contained in a picture is also unnecessary, and automatic detection of 
corresponding points realizes it. According to the singular point filter, luminosity and a 
position of a singular point can be maintained also on a level with coarse resolution, 
and it is very advantageous to object recognition, feature extraction, and image 
matching. As a result, a configuration of an image processing system which reduces 
help work substantially is attained. 

[01 14]The following modification art is also considered about this premise art. 
(1) With premise art, when taking matching between a starting point hierarchy picture 
and a terminal point hierarchy picture, automatic determination of a parameter was 
performed, but this method can be generally used, when taking matching between 
[ instead of / between hierarchy pictures ] the usual pictures of two sheets. 
[0115]For example, between pictures of two sheets, make 2, energy E 0 about a 
difference of luminosity of a pixel, and energy E, about a position gap of a pixel, into a 
valuation plan, and let these linear combination E^alphaEo+E! be overall evaluation 
types. Automatic determination of the alpha is carried out paying attention to near the 
extremum of this overall evaluation type. That is, it asks for map that E tot becomes the 
minimum about various alpha, alpha in case E, takes the minimal value about alpha 



among those maps is decided to be an optimal parameter. It is considered eventually 
that a map corresponding to the parameter is the optimal matching between both 
images. 

[01 16]What there are various methods in setting out of a valuation plan besides this, 
for example, takes such a big value like 1 /E v and 1 /E 2 that an evaluation result is good 
may be adopted. An overall evaluation type does not necessarily need to be linear 
combination, either, and should just choose suitably the n-th power sums (n= 2, 1/2, 
-1, -2, etc.), a polynomial, arbitrary functions, etc. 

[01 17]In the case of [ of eta and lambda ] 2, any, such as a case beyond it, may be 
[ alpha ] sufficient also as a parameter premise art. When a parameter is three or 
more, you make it change one [ at a time ], and it decides. 

[01 18](2) in this premise art, after deciding a map that a value of an overall evaluation 
type becomes the minimum, a point that C (m ' s) f which is one valuation plan which 
constitutes an overall evaluation type became the minimum was detected, and a 
parameter was determined. However, it is effective even if it decides a parameter that 
the minimum of an overall evaluation type only becomes the minimum depending on a 
situation instead of such two-step time processing. In that case, alphaEo+betaE! is 
made into an overall evaluation type, for example, a constraint which becomes 
alpha+beta =1 may be established and measures, such as treating each valuation plan 
equally, may be taken. It is because essence of automatic determination of a 
parameter is in a point of deciding a parameter that energy becomes the minimum. 
[01 1 9](3) With premise art, each resolution level generated four kinds of subsidiary 
images about four kinds of singular points. However, naturally 2 or 3 of 1 and four 
kinds of kinds may be used selectively. For example, if it is in a state where a bright 
point exists only one in a picture, a suitable effect should be acquired even if it 
generates a hierarchy picture only by f*" 1 3) about a maximum point. In that case, since 
a submap which is different with an identical level becomes unnecessary, it is 
effective in computational complexity about s becoming less. 

[0120](4) With this premise art, when one level advanced with a singular point filter, a 
pixel was set to one fourth. For example, composition which considers it as 1 block by 
3x3, and looks for a singular point in it is also possible, and a pixel will be set to one 
ninth if one level advances in that case. 

[01 21](5) When a starting point picture and a terminal point picture are colors, change 
them into a monochrome image first and calculate a map. A color picture of the 
starting point is changed using a map called for as a result. As the other method, a 
submap may be calculated about each ingredient of RGB. 

[0122][Embodiment about an image effect] An image effect device using the above 
premise art is explained. This device uses basic technology for recording an animation 
on a digital camera efficiently, and describes application to a digital camera henceforth 
[ drawing 24 ] after that explanation. Drawing 18 shows the 1st picture 11 and the 2nd 



picture 12 which are key-frames, and a certain pixel p, (x 1t y t ) and p 2 (x 2 , y 2 ) correspond 
among them. These correspondences are searched for with premise art. 
[01 23] Drawing 19 cuts a mesh on the 1st picture 11, and shows a correspondence 
position on the 2nd picture 12 of a polygon which constitutes the mesh. The polygon 
R1 observed on the 1st picture 11 now was decided by the four lattice points A, B, and 
C and D. This polygon R1 is named a "starting point polygon." these — four — a piece 

— a lattice point — A — B — C — D — drawing 18 — having been shown — as — 
respectively — the — two — a picture — I — two — a top — corresponding — a 
point — A — ' — B — ' — C — ' — D — ' — having — these — a point — forming 

— having — a polygon — R — two — " — a terminal point — a polygon — " — 
naming . Although a starting point polygon is generally a rectangle, generally a terminal 
point polygon is a quadrilateral. Anyway, by this embodiment, a pixel unit does not 
describe correspondence relation between the 1st picture 11 and the 2nd picture 12, 
but it describes those corresponding picture elements about a lattice point of a 
starting point polygon. The description is written in a corresponding-points file. By 
observing a lattice point, capacity of a corresponding-points file can be made very 
small. 

[0124]A corresponding-points file is used in order to generate an intermediate image 
of the 1st picture 11 and the 2nd picture 12. This point can generate an intermediate 
image of arbitrary time positions by interpolating a position of corresponding points, as 
premise art described. Therefore, morphing and a smooth animation between two 
pictures are generable to it by saving the 1st picture 11, the 2nd picture 12, and a 
corresponding-points file. For this reason, a compression effect of an animation is 
acquired. 

[0125] Drawing 20 shows how to calculate a correspondence relation about points 
other than a lattice point from a corresponding-points file. Since there is only 
information about a lattice point in a corresponding-points file, it is necessary to 
calculate matching information separately about a point inside a polygon. By drawing 
20 , correspondence with triangle A'B'C which hits a lower half of the terminal point 
polygon R2 similarly is indicated to be triangle ABC which hits a lower half of the 
starting point polygon R1 of drawing 19 . Suppose that the point Q inside triangle ABC 
by the side of a starting point polygon suited a position which divides the line segment 
AC internally to t: (1-t), and divides internally between the internally dividing point and 
point B to s: (1-s) now. What is necessary is just to think that corresponding-points Q' 
in triangle A'B'C by the side of a terminal point polygon of this point Q is in a position 
which divides line segment A'C internally to t: (1-t), and divides internally between 
that internally dividing point and point B' to s: (1-s). Namely, what is necessary is to 
divide a starting point polygon into a triangle and just to search for a point inside a 
terminal point polygon in the form of internal division of a vector about the triangle. If 
vector italic shows, it will be BQ= (1-s) {(1-t) BA+tBC}. 



Come out, and it is, therefore is B'Q - (1-s) {(1-t) B'A'+tB'C). 

Though natural, suppose that the same processing is made also about between the 
triangle ACD which hits an upper half of the starting point polygon R1, and triangle 
A'C'D' which hits an upper half of the terminal point polygon R2 similarly. 
[0126] Drawing 21 shows the above procedure. A matching result about a lattice point 
taken on the 1st picture 11 is first acquired like drawing 19 (S10). What is necessary is 
to perform matching of a pixel unit of premise art and just to extract a portion which 
hits at a lattice point out of that result at this time. However, a matching result about 
a lattice point may be specified based on others, and matching art, for example, an 
optical flow, and block matching, without using premise art. 

[01 27]It continues and a terminal point polygon is defined on the 2nd picture 12 like 
the right of drawing 19 (S12). Since a corresponding-points file is above generable, 
information for identifying the 1st picture 11 and the 2nd picture 12 is incorporated and 
outputted to this file (S14). Two pictures and a corresponding-points file are held at 
arbitrary recorders or media, or are put on a direct network or a broadcast wave, and 
should just be transmitted. 

[01 28] Drawing 22 shows a procedure which generates an intermediate image using a 
corresponding-points file. First, the 1st picture 11 and the 2nd picture 12 are read 
(S20), and a corresponding-points file is read (S22). It continues and correspondence 
relation between a point in a starting point polygon and a point in a terminal point 
polygon is computed by a method of drawing 20 (S24). At this time, a correspondence 
relation about all the pixels within a picture is acquirable, therefore, if coordinates and 
a color of a point which correspond each other are divided internally by u: (1-u) as 
premise art described, an intermediate image in a position which divides internally 
between the 1st picture 11 and the 2nd picture 12 by :u in time (1-u) is generable (S26). 
Not only interpolation interpolation but extrapolation interpolation may be carried out. 
[0129] Drawing 23 shows composition of the image effect device 10 which performs 
the above processing. The image input part 12 in which this device 10 acquires the 1st 
picture 11 and the 2nd picture 12 from external memory storage, a photographing 
camera, etc., The matching processor 14 which performs matching calculation to 
those pictures by premise art and other art. The corresponding-points file attaching 
part 16 which stores the corresponding-points file F generated by it, The intermediate 
image generating part 18 which generates an intermediate image from the 1st picture 
11 and the 2nd picture 12, and the indicator 20 displayed as an image near an animation 
of while carrying out timing adjustment of the 1st picture 11, the 2nd picture 12, and 
the intermediate image are included. The communications department 22 sends out 
the 1st picture 11, the 2nd picture 12, and the corresponding-points file F to another 
network and transmission infrastructure according to a request from the outside. 
Mesh data which show a size of a mesh, a position of a lattice point, etc. are inputted 
into the matching processor 1 4. 



[0130]By the above composition, the 1st picture 11 and the 2nd picture 12 which were 
inputted into the image input part 12 are sent to the matching processor 14. The 
matching processor 14 performs matching calculation of a pixel unit among those 
pictures. The matching processor 14 generates the corresponding-points file F based 
on mesh data, and outputs it to the corresponding-points file attaching part 16. 
[0131]The intermediate image generating part 18 responds for a user to ask, or reads 
the corresponding-points file F by other factors, and generates an intermediate image. 
This intermediate image is sent to the indicator 20, timing of a generating picture is 
performed there, and an animation or a morphing picture is displayed. This device 10 
may be in the remote terminal side of another composition, that terminal receives 
comparatively light data called the 1st picture 11, the 2nd picture 12, and the 
corresponding-points file F in that case, and the intermediate image generating part 
18 and the indicator 20 can reproduce an animation independently as this operation 
also shows. 

[0132]The communications department 22 is the composition for it, and it assumes 
that a remote terminal has already waited at the network point here. That is, the 
communications department 22 puts the 1st picture 11, the 2nd picture 12, and the 
corresponding-points file F on a network or a broadcast wave, and sends them out, 
and a display of an animation is performed by the remote terminal side. Of course, a 
remote terminal is good for the purpose of not a display but storage. This device 
inputs the 1st picture 11, the 2nd picture 12, and those corresponding-points files via 
the communications department 22 from the exteriors, such as a network, passes 
these to the intermediate image generating part 18, carries out interpolation 
calculation, and also has a use which generates an intermediate image. The course P 
of data for it is described by drawing 23 . 

[0133]When experimented about the contents of processing of an embodiment, very 
good morphing or an animation compression effect was acquired by adopting about 
256x256 size, for example, and setting a lattice point as every direction 10 - every 
tens of pixel grade as the 1st and 2nd picture. A size of a corresponding-points file is 
about 10 K bytes from several kilometers, and it has checked that height of image 
quality and smallness of data volume were realizable. 

[0134][Embodiment of a digital camera] Drawing 24 shows composition which applied 
the image effect device 10 of drawing 23 t o the digital camera 50. Hereafter, it 
explains focusing on a difference with drawing 23 . In drawing 24 , it changes into the 
image input part 12, the image pick-up part 52 is formed, and the camera controller 54 
which controls this is put side by side. The IC card control section 56 is formed as the 
corresponding-points file attaching part 16, and this controls input and output of data 
to IC card 58. Altogether, via the IC card control section 56, the 1 st picture 11 , the 2nd 
picture 12, and a corresponding-points file are constituted to an IC card so that writing 
is possible. Although an output to a network, an external storage, other external 



transmission medias, etc. is possible for the communications department 22 in the 1st 
picture 11, the 2nd picture 12, and a corresponding-points file and it is the composition 
of receiving data from the IC card control section 56 in the figure, of course, that may 
not be right and composition received from the usual data bus may be used. The mode 
setting part 70 sets up photographing mode by the camera controller 54, and 
specification of "simple animation mode" characteristic of an embodiment besides the 
usual still picture mode and animation mode is possible for it. 

[0135] Drawing 25 shows an example of the image pick-up part 52. An image is 
acquired by CCD(charge coupled device) 60, is digitized with A/D converter 62, and 
receives processing of a white balance etc. by the pretreatment part 64 before record. 
The 1st picture 11 and the 2nd picture 12 of the last form are recorded on IC card 58, 
or receive processing by the matching processor 14 directly. 

[01 36] Drawing 26 shows example of another of the image pick-up part 52. Unlike 
drawing 25 , a fixed distance is kept, 2 CCD60 are provided, and photography of a 
stereo image is possible. A/D converter 62 and the pretreatment part 64 process an 
image from 2 CCD60 by time sharing. However, also as for these, successive 
installation **** is also good 2 system. 

[01 3 7] Returning to drawing 24 , processing of the camera controller 14 is considered 
as it goes, and it gives an example below. 

1. Photo the 1st picture 11 and the 2nd picture 12 with a predetermined time interval 
(only henceforth a "photographing interval") by the image pick-up part 52 in one 
photographing operation in the digital camera 50 which adopts ocellus composition of 
use drawing 25 w hich compresses an animation as an ocellus camera. At this time, the 
digital camera 50 is in the above-mentioned simple animation mode, i.e., interim 
photographing mode of a still picture and an animation. 

[01 38]If a release button is pushed in this mode for a user's photography, a picture of 
two sheets will be photoed, for example for a photographing interval in 1 second. When 
a photographic subject moves in this 1 second or a user moves, a difference comes 
out between the 1st picture 11 and the 2nd picture 12. The matching processor 14 
generates a corresponding-points file in order to bury this difference. An animation 
corresponding to for 1 second is generable because the intermediate image generating 
part 18 generates an intermediate image based on this corresponding-points file. If 
duration time of a reproduction animation is set as more than 1 second by generating 
many intermediate images, a slow motion picture will also be acquired. 
[0139]A generated animation is displayed on the indicator 20 by a liquid crystal device 
etc., and the user can check the contents of the simple animation. Of course, the 
indicator 20 may display only the 1st picture 11 and the 2nd picture 12. In any case, 
since a corresponding-points file is recorded on IC card 58, the animation can express 
as apparatus of the exterior of the digital camera 50. The external instrument shall 
have the same composition as the intermediate image generating part 18. 



[01 40]If a photographing interval in this mode is lengthened, though natural, an 
animation of longer time is generable. How much time is allowed should just determine 
by a relation with image quality, and a user may set up. Exposure time may also be 
decided by the mode setting part 70. 

[0141]2. If a photographing interval of the use above which generates a morphing 
picture as an ocellus camera becomes above to some extent, approach generation of 
a morphing picture gradually rather than an animation. Therefore, it is good also 
considering it as specification of the digital camera 50. In this case, further, a view of a 
photographing interval may be stopped and a user chooses the 1st picture 11 and the 
2nd picture 12 with the function of the camera controller 54. Those pictures may be 
pictures which could choose from an already photoed picture and were inputted from 
IC card 58. Anyway, morphing is possible also between pictures which are completely 
unrelated in that case, for example. A morphing picture in which an experiment is also 
very good is generated. 

[0142]3. In the digital camera 50 which adopts compound eye composition of use 
drawing 26 which generates a multiaspect image as a stereo camera, photo a picture 
of two sheets simultaneously and the matching processor 14 generates a 
corresponding-points file. In a corresponding-points file, corresponding points in the 
2nd picture 12 photoed by CCD60 of another side are described about each point of 
the 1st picture 11 photoed by one CCD60. Therefore, depth of the point becomes 
clear by a principle of triangulation from a horizontal gap of a point (henceforth "a 
corresponding-points pair") that those pictures correspond each other. As a result, a 
special effect image is generable by processing of emphasizing depth. 
[0143]According to the intermediate image generating part 18, a view image between 
both CCD60 is generable. If extrapolation interpolation is carried out, a picture from 
not between CCD60 but a viewpoint distant from the digital camera 50 to some extent 
is generable. A multiaspect image is also obtained by assuming various viewpoints. 
This also becomes a basis of a walk-through picture. 

[01 44] Both CCDboth [ one side or ] may be provided removable, and it may constitute 
so that those air clearances can be taken to some extent. In that case, performance 
as a stereo camera is improved. 

[0145]In the above, an example of a digital camera concerning this invention was 
explained. Although set centering on a personal youth's camera here, this invention 
may be applied to a camera carried in a professional youth's TV camera and an 
artificial satellite. 

[0146]Like drawing 23 , the digital camera 50 inputs the 1st picture 11, the 2nd picture 
12, and those corresponding-points files via the communications department 22 or IC 
card 58 from the exterior, may pass these to the intermediate image generating part 
18, may carry out interpolation calculation, and may generate an intermediate image. 



DESCRIPTION OF DRAWINGS 



[Brief Description of the Drawings] 

[Drawing 1] The picture and drawing 1 (c) which are produced by drawing 1 (b) giving 
an average filter to two persons' person's face, and drawing 1 (d) with drawing 1 (a), 
the picture, drawing 1 (e), and drawing 1 (f) of p C5,0) which are called for with premise 
art about two persons' person's face, the picture, drawing 1 (g), and drawing 1 (h) of p (5, 
1) which are called for with premise art about two persons' person's face, the picture of 
p (5 2) called for with premise art about two persons' person's face, and drawing 1 (i) and 
drawing 1 G) are the photographs of the halftone image which displayed on the display 
the picture of p C5, 3) called for with premise art about two persons' person's face, 
respectively. 

[Drawing 2] The figure, drawing 2 (A), drawing 2 (B), drawing 2 (C), drawing 2 (D), and 
drawing 2 (E) which drawing 2 (R) shows the quadrilateral of a basis are a figure 
showing an inheritance quadrilateral, respectively. 

[Drawing 3] It is a figure showing the relation between a starting point picture and a 
terminal point picture, and the relation between the m-th level and the m-1st levels 
using an inheritance quadrilateral. 

[Drawing 4] It is a figure showing the relation between the parameter eta and energy C f . 
[Drawing 5] Drawing 5 (a) and drawing 5 (b) are the figures showing signs that it asks 
for whether the map about a certain point fulfills bijection conditions from outer 
product calculation. 

[Drawing 6] It is a flow chart which shows the whole premise technical procedure. 

[Drawing 7] It is a flow chart which shows the details of S1 of drawing 6 . 

[Drawing 8] It is a flow chart which shows the details of S10 of drawing 7 . 

[Drawing 9] It is a figure showing the correspondence relation between a part of 

picture of the m-th level, and a part of picture of the m-1st levels. 

[Drawing 10] It is a figure showing the starting point hierarchy picture generated with 

premise art. 

[Drawing 1 1 ] Before progressing to S2 of drawing 6 , it is a figure showing the 
procedure of preparation of matching evaluation. 

[Drawing 12] It is a flow chart which shows the details of S2 of drawing 6 . 
[Drawing 13] It is a figure showing signs that a submap is determined in the Oth level. 
[Drawing 14] It is a figure showing signs that a submap is determined in the 1st level. 
[Drawing 1 5] It is a flow chart which shows the details of S21 of drawing 12 . 
[Drawing 16] It is a figure showing the action of energy C (m s) f corresponding to f m,s) 
(lambda=ideltalambda) called for while changing lambda about certain f m s) . 
[Drawing 17] It is a figure showing the action of energy C n f corresponding to r n 



(eta=ideltaeta) (i= 0, 1, — ) called for while changing eta. 

[Drawing 18] It is a figure showing signs that the pixel which it is between the 1st 
picture and the 2nd picture corresponds. 

[Drawing 19] It is a figure showing the correspondence relation between the starting 
point polygon taken on the 1st picture, and the terminal point polygon taken on the 
2nd picture. 

[Drawing 20] It is a figure showing the procedure of searching for the point in the 
terminal point polygon corresponding to the point in a starting point polygon. 
[Drawing 21] It is a flow chart which shows the procedure which generates a 
corresponding-points file by an embodiment. 

[Drawing 22] It is a flow chart which shows the procedure which generates an 
intermediate image based on a corresponding-points file by an embodiment. 
[Drawing 23] It is a lineblock diagram of the image effect device concerning an 
embodiment. 

[Drawing 24] It is a lineblock diagram of the digital camera concerning an embodiment. 
[Drawing 25] It is a lineblock diagram of the image pick-up part of the digital camera 
shown in drawing 24 . 

[Drawing 26] It is another lineblock diagram of the image pick-up part of the digital 

camera shown in drawing 24 . 

[Description of Notations] 

1 0 Image effect device 

12 Image data input part 

14 Matching processor 

1 6 Corresponding-points file attaching part 
18 Intermediate image generating part 
20 Indicator 

22 Communications department 

50 Digital camera 

52 Image pick-up part 

54 Camera controller 

56 IC card 
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(SI 8) 

Cm - s> f cm) d. j> ^wsftfcttntf&s 

[0 0 4 9] 9clCfemLrc3-*)\/3r— E Q «\ |ml/^ 
/Wc43«-3BI¥«f Cm. °) ^l+^-Ti.fc46{c, ^ 



£fc, ffll¥«f 



cm. s) ^im-t^rctbicit^^m 



[0 0 5 0] 
Kl 9] 



[0 0 5 1] 
[ft 2 0] 



(55 1 9 } 



= n/^c*, j) - / (m ' s - 1J (i,y)!i 2 (i < o 



cm. s) (i, j) t ^ mm- 1 \y^^<Dmm<osi 

[0 0 5 2] (SIC, ffl^rafflJBA' B' C D' OrtSP 
$?\ A' B* C D' ©«m^fr5©SESi 



U20) 

fef, CMf Cm. s > (i, j) OffifcLTStR-T 

OL Cm) ma xtcPJJi-rSST*. L*^f <LT^ 
<„ L Cm) maxtigU^mCSLta^ffeSo 

[oo5 3] &mmm&*m^%i&nmt, m%m®m 



(9) 
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&*^rc-?ztiz^vimx$>2> a mm<ov 

^X«tfr46T/h£&fe<DfcPI^L&tfn(£&£-f\ ^ 
ftO/hS&ilHgiLM&dc £tfT*%%\,\ 5 etc, 

[0054] [i. 4] mm&^u*-*m<D&W)fti£ 
/^?<—$m*^ic&W]Vt.m-?z>z£tfxzz>o 

[0 0 5 5] mmkffifcfa%*y7,7-KiZ&rc-D(r)^ls* 

xRxiri^ts. %&mc^z.i£^ ximmcomm 
<DmcDmz>-x&o, Tjfi^fs^pjtt^-^LTi/^o an 

5) ©;^7 y - ^ 0 Tfe O , Sf 5 tj = 0lc 

[0056] i. **mt^t-pa^*ii:H 
t>\ mmmt>m^£^orctfxm^xttfo-z>ift>ti 

2. ^©fc^ MM?? LcDftfcMm^frL 
[0 0 5 7] 3. ^C0*S^ iU 4fC*5^TD 



i6 s D to- f 0S^ig*n^fflS<Jt-§<i:5 f 
cm. s> ft^tU tco^mc to. S ) f tfmimir 

hfctfiX, A^tg/)n?H±^^e^l 4^«/J^i^<!;?) 
tl^ttl^»LOOC to. s) f ^/>^C,tgjf,p 

fc$ci;i>igffl^m u ^© a ^ tj = o {crjotfasiifi 

<fT^ 0 of£tj^>L^O^LtC to. s) f£ D 
[0 0 5 8] CcDTrSHaU AP^©*i^>X^©lU5$l 

g l & # e ££tm B(omm<D-?v^ r& ten 

[0059] [1. 4. 1] A ©ftW&S 

^^.jetgijw^wfffi^n^o 1 4"©c"«t < „ s&x* 

M'-iiAC to. s) f +D to. s) f fc,i;-pTS 

t?n§o ^9©d to. S ) f temt>fr-z*m-?$>(D 
x\ mmmctemzmMi'Dm^icm'hictev . mmm 

t?J5 if E 0 E ifctiiPLTVK. E ! {£iltfcT?&i>fr 

D Cm. s) f £D*/hglJ^lHtt 1 T$>§ 0 COfc 

46. Il&OAC to. s> (i J5 o^{t («iJ>*) 

# i w±-e&w-n«r, m$i*mitmi%£ tiz&^xm 

ffttffoTD to. s) f l4 i W±iiAp-rsfc:46, A 

c Cm - s 5 < , . j ) # i Jeuh*^Lav»iB*)ttx* 

« ft 6) 4 V 46 T § o 

[0060] ncD^co^t, xcomtaicm\ JEn& 

m&lcc to. s) C1 . 5 tfj]£'>-f ^ct^-To 
c Cm - s) ct. j) ©t:^^^^ ; &h (1) tie 
kR-T^o h (1) &x*;l^-C to. s> Ci J? 
3^ 1 2 T-S>^B^cDi^-t T -fe§c, A 1 2^ i jfm.K)\L-Z>rc 
46tc, flW^ttfl 2= l/A©Jg^%#*S 0 A^A x fr 

5 a 2 s -e«/jN«^tr a t 

[»2 1] 

= k w dx 

(521) 
[12 2] 

(552 2) 

SiifiLTW. cootie to. S ) f cr)fii^ 

[12 3] 

(S23) 

[»2 4] 




(10) 
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ax 



MO 

A 5 /* 



(5S24) 



tfl£±Lir% a h (1) >0Tfei^6, iimc 

±jScd^, osoc cm. s> f oti 
ftp^aE-ri»o ^ostti^ffi-f ^c^c^o. aoi 

[0 0 6 1] H (h>0) Wk^WLtt^t 



[S2 5] 
[15(2 6] 

ax ~ a 5 /»+*p 

[^2 7] 



(£2 5) 



(5* 2 6 ). 



(3/2 + Jfc/2)A 3 / 2 +V2 



U27) 



CWC Cm. f to— ISSTfe^ (C«5£ 

if) o 

[0062] \<Dmmmztmtti-?%P%. s^c^^n 

P o i:{S^-rSo 
[1*2 8] 

< a28) 

[0 0 6 3] 
[^2 9] 



MOpd 



C5S2 9) 



= 1 



(S30) 



lot, 
0&3 0] 

BpX arl 

MM 

liSSiT^^o {g^ch (1) =H1 K^jg^-r^^^, 
[S3 1] 

BoX 3/2+k/2 = fb/f (St3 1) 



3/2 + k/2/ 2 mcr^l^g^ b 



O t h r e s 



safiiti, x^-^-c cm. s> f commTizmm ls 

[0 0 6 4] ^*5^T?«, Rl]^fg!f Cm. s) ^§f^: 



fSBR* tLA^O. l*Mz.fct>f Cm. (DltSE 
«±i6Tf C m - s + D (DMmicWnLrco A>0. 
1 IfflHOilElJg 2 5 5 U^^HfOtJ-r^ r3j © 

m^t>^mmmm^wm\,rcTcibi:^>K) . a>o. i 

[0 0 6 5] [1. 4. 2] tXh^"^Ah (1) 
C Cm. s) f co&mZtT, h^7ih (1) tC#c#L 
^I^S^CD^ 3 cD^cD^fico^ h (1) 

icmmz&fvZo mmc a, c cm, s > f ) 
uvh-?%t, kirmn i ^3fitc$,i, 0 ^gn?tek = i 

cDffl^ 1 ^-e$>tii£, B 0 A2^ Bl A2 nmmczs: 
h-f, 77^^-A ci-k) /2 [C f ot ^ (C {gj, D 
■T^o h (1) ffl£&."e&til& M^(f7 7-^^-«A 

[0 0 6 6] d £ T^S:© C* i: < &.£iiHt!£ f>L>tf ( x 
[S3 2] 



p,. n = / ^v^'-^ + tf-w)') (VP- + < •■) 

\ 0 \atkerwitt) 

(S3 2) 

-73. i&tmmt. ^<DCt <w(x P y x ) , 

[S3 3] 



(ci/iertci*:) 



(s£3 3) 



CCT'c (x) t±c (x) =xk©$T'^mo * 
& (x 0 , y Q ) M(x,, y ! ) ^+^iSl^^ 
H^h^Ah (1) irj&Z<DBt%:2> 0 
[S3 4] 

w v 7- / (S3 4 ) 



(11) 
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TAD^TVx^ h^^^t^i. fc#x.Tk£ 
[0 0 6 7] fc*J x ^3 4*^*5^S«fc5fC rttiHfc 

scttcaac-r^fr'T**,, cot-c^k [1. 4. 1] 

[0 0 6 8] [1 . 4. 3] TjtDSjfl^g 

7] = 0 1 U «fc*ffl^i/^#Sfcfcnt?.«^W^¥^ f 

[0 0 6 9] 
S3 5] 



(S35) 



Jl^OOtt, D cn) f &iSM©giJ3?«i:J!8ll8«fci* 

fit), YfifitiC ftSS. 
[0 0 7 0] dCD^jSTC (n) f £r«/NC"r£«31& 

S<«6ff«:*<6«{tr*o A A2ltf»/h« 

if 5 *^fiPffit!pJ»f'r S c i: «/Jvfi©«Si 
Jlofrftfcf v £ 6 Jciffl^v^Effl^K^-r § c i: <fc o 

[0 0 7 1] [1. 5] X-^^'J^^* 

^ f cm, S ) comm^R x Rtcngg-rsc twez 

[S3 6] 



(S3 6) 

cm. S ) ttgEstR^gfafli^isctft^sn, 

[$(3 7] 



^(9(.\i)+(0.5,0.5)) 



(S3 7) 



[8(3 8] 



(S3 8) 



[0072] [ 1 . 6] &mm<Dmm<Dmm<DiEm<t 

§:n*;l/^-C Cm. s > r i^f <ftotf, JELi^ 
[0 0 7 3] flRfcf. AcOMtSaCOMO^-y^^^t 

^^-&^#^.i.o ffi<DmittT°mt>nx*5*), intern 



[0074] [1. 7] -i'y7"y>(yf->3>' 

(i, j) = (0, 0) fCOV^Tf s ^ tDfil^^ 

Vkxc i * 1 -r o«-»f>La*^«- f 

Cm. S ) ( i? j ) C0fiI£:&5£-r3 o i Offl^iS^O 



(12) 
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[0 0 7 5] feSp (I> KO^T^Mq 

f C i . j -+- O * s ^*5>tlSo CGD^, q 

tfSKfcSo otetc (o, 0) tf«fcflBfcSft**aR# 

z>o *Ki«SII5if«co«l8*@jB-r*fc«>fc, f 
Cm. s) %iXT©73j£T&a6T^< 0 
[0 0 7 6] (s mod 4) **0©J§£\ (0, 

o) ^MtejSfcL iRtf j ^^tc*-^ barest at 

^ < 0 ( s mod 4 ) # 1 «±ff <DftiQ&* 

<o (s mod 4) *"?2©fc^ v «Ttf©*r4S^M 
tejStU i &tf j *Sfci>S-&fctf5ft«>TV><o (s 
mod 4) tf3©J§-£\ mTft<D&t%£L*M1&&t 

tc.fc.LsC C T\ 



4 — r, (m '* ) J" 1 -*) 

[8 4 1] 

^ - ?/<»..) W _ l) ?(fc,Z) 

(3C4 1 ) 

T&& (uu^^hMH^^b/l/tU zftht* 

itS!^ffi«3Rtcfev^T^asns) 0 ttw^t-es 

tltfx ^©^{COV^TtiD Cm. s > Ck j-, 

[0 0 8 0] 05 (a) , 05 (b) ttCOift^ 
■TSa**^LTV^ 0 05 (a) lt^i-^y-^<D^ 

mm, 05 (b) (^t^f^^iti^fnfni 
•to mmmm ( i , j + o »c»-rs^«f cm. s > 
( i , j + 1 ) &mm? 5^ wo z ^^T^n« 

q Cm. s) (k 0 ti|^^-r 
[0081] [1. 7. 1] sm{%i(DM!¥ 
tfaio (0) =0, o (1) =K o (2) =2, o 



stfft&L&^rctb, {gtc s = 0Rtfs = 
2 T&5 LT*rcOO^|«ilJ&^WKH-ilLfco 
[0 0 7 7] ^RO-rv^y^^r— v'a^Tli, 
^^aSS^ffiK^LT^^U-rw^ASCfctcJ: 

m (k, i) ©+fr&^#*iB 9 

/fTf Cm. s) (j, j) ( m =0, -, n) 
3g/uf£ 0 lg3^fr£®£^*f©x*;l/:£-D (k, 
1) -73. SI SrcttS20*fr*»S 

Mffi fcf+ 0 %mi& a 0 = 2, ^= 1 00000 

[00 7 8] M5zE<D^*r^#©^x^^©rc46{C, * 
l^©¥^£LT (k, 1) =f Cm. s) (i, j) ^ 

cm. s > ( j , j ) <Dmwmmm\rL^n2>&te : ?& 

(k, 1) tCttU WO^lOzlS^OWltftS 

[0 0 7 9] 
[83 9] 



(5S3 9) 

[84 0] 

(340) 

(3) =3, o (4) =0*fflV\ iS8cDi:#iio 
(0) =3, a (1) =2. o (2) =K a (3) = 
0, o (4) = 3£fflVV£«, CcDd^T, 

K>y^yy;vLfc 0 it is. *&wm*t* 4 mmr*& 9 , 

s(i0~3OV^-rn^T£^o LfrU H^lC«s = 4 

[0082] [i.8] mmm 

Ci. j > P Ci + l. j> P Ci. j+O P 

c i + i . j + o *^^B« J F®±^e32^q 

f Ci. j) (1+1, j) qf Ci. j -+- 1 ) Q 

f c i + 1 . j ■+- 1 ) fcM&znz tm.j£-?Zo m^.(D 

gttft (0<t<l) ?&Z>rpmWWl<Dmmr (x, 
y, t) (0^x^N-l, OSy ^M- 1 ) 
BffiT3R»5nSo S-riSJRr (x, y, t) co{4H 

(fcfcLx, y, t£R) ^Mt'*»5 0 

[0 0 8 3] 

[84 2] 



(13) 
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(«,») = (l-«fc)(l-rfir)(l-t)(i,i) + (l-cfe)(l-<fc)i/(i,i) 
+ dxp - dy){\ - t)(i + + dx(l ~ <h/W(* + 1.3) 
+ (1 - dx)dy{\ - t)(i, j + 1) + ( 1 - dx)dytf(i, j + I) 
+ dady(l - + + <2«i»l/(t + 1,3 + 1) 



(5£4 2 ) 



O-^VTr (x, y, t) (Cfctt§ia^^il?je*^0!S [0 0 8 4] 



+ dx(l - dy)(\ - i)V(p(i +lJ )) + <Ml - dyW^qju+tj)) 
+ (1 — <fs>*/(l - 0V(p(k*i)) + (I - ^)<^«V(^, iJ+1) ) 
+ ciirfy(l -*)V(> ( . +u+1] ) + dx < ijrfV(^(j + ij + t)) 



[0085] [1. 9] ms&$kft*mLrct%<Dm% 

[0086] S-r^*i®#co^o® 



W43) 

[0 0 8 7] S-f^46t, ^H^^Offl^^*!.^ 

[0 0 8 8] J<$:fr%:mSlFteUT<DmffiT&iV>2>o S 
[S4 4] 



P(*0v?o) ' P(*l*7l): 



U4 4) 



[a 4 5] 



(3i4 5) 

f&&mm<Dit&<Dmm<D&{ii.mii, Pan. j h > (h ©OToaiist*f»stis, 

= 0, n s - 1 ) ©&ffifc£*ttW-*LT3R«>e>n [0 08 9] 



(34 6) 



fctcLCCX, 



m4 8] 



[»4 7] 



* K J ' total weighty, j) 



(S4 7) 

total weighty j) = £ l/||(s* - (5S4 8) 



h=Tl.-l 

h=o 



(14) 
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[0 0 9 0] o-^VT, F Cm> fcjfiv^M^ffll f #«fc 



[0 09 1] ci!T% f Cm. s > (i,j)#F 

c™> (i,j) iC+^j&^t^. -3%iD*ttlt><Dmffi 

[S5 1] 

[ 2 2(^-m)] (351) 

S-f Cm. s > (i,j) tfF cm) (i,j) tc+^ifiV^IE 
M^iKWlc&a&fcv^ci&TfeSo coiitiO, IE 

[0 0 9 2] [ 2 ] *f*WfcM3MH 

[i] ©#S3fefttBtc<):sftyio»in*siwr-5o 0 6 

m*fi^ (Si), o^T&j&H«fc*l£H#©vy 
(S2) o fcfcU S2liMT'ttft( 1 

s i ■e»e.nfeHi«oi#a*t»fcn:a«BiBaii*ifo«!.3i 

[0 0 9 3] 0 7«06CDS 1 <Dg¥£ffl£-^-f 7 n— 
-htM. LctiiS 2 T?&j£H« fc*&&H«<JD v >y 

S*7^;b^fc*oT^iii^|igS<k%tfv^ (s 1 

0) „ -jl<D^Pg«ja«%^*o OC>VT|W|«<D?3?£ 

T?»^iBfliioBS<t%fTV^ (s i i) , -a«*i*i5§a 
wm.*%z>o fc/cu siotsi i <Dmm£&MT& 

Z U ft&jSPBSiffi«i:»^B«iiii«%MtTLT±ritLT 

[0 0 9 4] 0 8tiE7<DS 1 0©ii^-r7n-f 
t-hT^5o &£<D&*M£>-9--rXli2 n x 2 " 

-^m^rntc-fe-y hTZ> (S 100) „ O^^t^mU 



^-D Cm. S ) 0 j5 ^It^o jEffitcWu D 
cm- Cl> j 5 « x 

[»4 9] 



-s;l/CDlli#p Cm. O) % p Cm. 1) ^ 
p Cm. 2) % p Cm. 3) ^f>Hl*7^;W^ffl^ 

TwajS**tfj.b (sioo, ^nfnim- 1 1^ 

;l/(Dp^p Cm— 1 . 0> % p Cm— 1 . 1) „ p 
Cm- 1 . 25 ^ p Cm- 1 . 3) (S 1 0 

2) o d ^T(±m= n V$>&fcib, p Cm. o) = p 

Cm. 1) =p Cm. 2) = p Cm. 3) = p (n) 

[0 0 9 5] 0 9(iBmlx^;l/£D@i#©— gPi:, glm- 

»te«#iiiii&©»e£js-ro pia©p cm. s> «p 

Cm. O) ~ p Cm. 3) <D\^<DmWkW&$ : b%><D 
T\ p Cm-l. o) P 
Cm. s) t ip Cm. O) t'$^i#X.§„ [1. 2] 

lft<D?*> T3J , p Cm-i. o & rsj , p 

Cm- 1 . 2) ^ T6J , p Cm- 1 . 3) ^ Tl 0J 

cDSiJH{t^-9-^X«2m- i x 2 m ~ i tC&£ 0 
[0 0 9 6] OO'^Tm^U^VM (08 OS 1 
0 3) » m^tcaoTV^l/^dt^fitSL/ (S 1 0 
4) % S 1 0 1 fcjgoTorfc»fl^©fflV'>l!liii«*£ 
fi5cLTV>< 0 ClOllOigUeyiOlgS, m=0, "T^to 

^|go^;i/®iiJii«*^ricsnfc«FjS-es i oamr 

■T^o 3f 0 l^^l/OffliB«©-9-i'X« 1 x l -pftS. 
[0097] 010 li SI 0tC<feoT^$n/c:{!&^ 

08£Dsaat±07os i lK&^aiT&tK ra«© 

[009 8] ffiffiSWeHu 0 6 © S 2 tcjitrfciftfc v 

^5. |B]Hori:<, ^^Ifi^OHTfithxWsSJg^ni, 



(S49) 

[»5 0] 

_ / 0, if l|FC->(i, j) - ^KiJW < bfeJ 
~\ \\F< m ){i,j) — /(™'*>(*,i)|| a ) "otherwise 



(S50) 



(15) 
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(S30) o [i. 3. 2. i] r-mALfcMmicm? 

%3L^;l¥-C Cm. s > e £ [1. 3. 2. 2] T2> 
ALfc«om?)^^tcWr§x^;^-D Cm. 

II (AC Cm. ( i . j j 

• ^ (1 , j>) (SC5 2) 

t45 0 fcfcu mmt i , j fcov^r^n^no, i 
- . 2^-i -r-ttmr s o «±t? v >y ^ > ymmwmm 

[0099] 01 2[i:06OS2 <E>SMffl*aVr:7 D-^ 
t-hT?$*. [1] T3zE^rcC'^<. te£l8j|Btt& 

**,&Pgjf BHfiC^ >y ^^ftswciei i;^jgU'^;i/£0 

43 cfc tf » j£IBJI1 h N»%£j£ L T t fci6, <ftaS©ffiH 

0, ^W^V>y9 L >^Oiem««^it^Tl^(cS 

[0 1 0 0] 0 1 2<D<I*fc<, S-r«»^7^-^ii* 
0, U^;l/^7^-^m^0K:^t5 (S20) „ O 

SS&SPgJiifflfl^O^m l^;l/© 4 OtDHIBftO^n^ ft 
f Cm. s5 ( s =0, 1, 2, 3) ^T5f<46i. (S2 

i) o ±%LM0kmz [i. 3. 3] x^rcmmmmm 
?:fflv^i$n§o hois, sn, i 8*^-r<k^ 

v -y f - y vim&fflm jntv^o c n«^& s u-oi/ 

HOliW#jH7*5, &*5 V V^nr^OTfe-^T^ft 

[0 10 1] —7?. |WJ — U-^;brtfc43lt57j<W#BS«, 
fifrft&o [1. 3. 3] £>iS2 0©cr£<, f 

Cm. 3 5 ftf Cm. 2) f Cm. 2) ft f 

Cm. O fC N f Cm. O ftf Cm. 0 5 f C> ^tl^ 

iott, *-ft 1 1> t m vith&Mm t te&wtoK. 

3 Ltfifi^ft ifx^;l/^-ft/jN$ < & 0 . v >y ^y 

[0 10 2] fc*3, fi$JfC&#>£^ f Cm. O) {CO 

l 9fc^c"£<t>£o)|g^l^;l>^#Brr£o fc/c 
U H»^ftf cm- 3> cn^M 

*fttuf cm. o) ■^-^mm-^^h^o^-wth 

ofco ilfttta2 0ICs = 4^UL, f Cm. 4)^ 



■&MSC*3iT5 (S3 Do [1 . 3. 2. 3] 
ALfc*£x*;l/^-A C Cm. s5 f + D Cm. s) f 
tfZtlT'&O. [1. 3. 2. 2] -r-JiALfcTj^ffl^ 

7] E o Cm. si (1 j , + E ! Cm 
fffc&f Cm. O) fc-rSCfct^LV^o f Cm. 0 5 

tf cm. 35 Qn«^<&Dii3f«fSlRl&l§iarr 

&ofe 0 c©itBi«:*n* % ^Ufc-efc* [1. 7. 1] fC^ 
TlWffli©^* -y 7;l/fefTo/c 0 

[i. 7] -ea^fc^fcD-e&^o 

[0 10 3] HI 3li.mo\y^Mz^XW\m^\k^. 

■r^m^^.tmx$>^ a mo ^Dwn&mmmfirc 

f co. s5 at<tiiwtfSf®cfti§o 01 
4 ft^ i is^Mc$5^rmm%%:&m-?z>m : ?z^-?m 

TfeSo ^i ^;i/T*«Blia«^ J E-n j Pti4iiilS-e*^ 
£ti& 0 raH-e««:nc. 4@i*^ii-c t -^snTv>^ 0 

p ci. s5 <D&x(Dttfo&%:q ci. s5 CQi=f=i 
[0 10 4] 1 . mi ls^>\s<DM&&T\*x<D%.±& 

2. ^a-d^toffl^l/^k D^0 lx^<;l/tc 

n^'nii^A-Dtcs-r^o fcf£L, aiigA~cft** 

3. mo U^;l/T-TTtC*$oTV^a^A~DCD^ 
,6A' ~D* ^rq Ci. s5 (O^Cfu >y f f i>o ISlii 

a' ~c o > ^n^nsi^A-c 

£mw&mfc$>%>$><D£?%> a 

4. ilAOWaOMSa' ^HKIA* O^fcfe 

tm&h' <D$>T<£tb%ikmtfmVT&2>£fol£irz 0 

5. 4£ mm<DJi&VMfo& b 1 ~ d ' ^7*P7hU 
*a' ~d' T*B«E3ia«*f^So 

6. fflgEEgia^©>t'Tx^;l/^-^«/Nc^:SJ:-5, 
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